-- Logs begin at Mon 2022-09-12 22:49:08 UTC, end at Mon 2022-09-12 23:46:09 UTC. -- Sep 12 23:13:33.889197 dv-cinder-852009-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 12 23:13:33.896505 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 12 23:13:33.898504 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 12 23:13:33 2022] *** Sep 12 23:13:33.898504 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 12 23:13:33.898504 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 12 23:13:33.898504 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: nodename: dv-cinder-852009-7-iscsi Sep 12 23:13:33.898504 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: machine: x86_64 Sep 12 23:13:33.898504 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: clock source: unix Sep 12 23:13:33.898504 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: pcre jit disabled Sep 12 23:13:33.898891 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: detected number of CPU cores: 4 Sep 12 23:13:33.898891 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: current working directory: / Sep 12 23:13:33.898891 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: detected binary path: /usr/bin/uwsgi-core Sep 12 23:13:33.898891 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: your processes number limit is 47680 Sep 12 23:13:33.898891 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: your memory page size is 4096 bytes Sep 12 23:13:33.898891 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: detected max file descriptor number: 2048 Sep 12 23:13:33.898891 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: lock engine: pthread robust mutexes Sep 12 23:13:33.898891 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: thunder lock: enabled Sep 12 23:13:33.898891 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 12 23:13:33.899320 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 12 23:13:33.939796 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: Python main interpreter initialized at 0x55d2cd2979c0 Sep 12 23:13:33.939796 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: python threads support enabled Sep 12 23:13:33.939796 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: your server socket listen backlog is limited to 100 connections Sep 12 23:13:33.939796 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: your mercy for graceful operations on workers is 90 seconds Sep 12 23:13:33.940130 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: mapped 403077 bytes (393 KB) for 2 cores Sep 12 23:13:33.940252 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: *** Operational MODE: preforking *** Sep 12 23:13:33.940529 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: *** uWSGI is running in multiple interpreter mode *** Sep 12 23:13:33.940529 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: spawned uWSGI master process (pid: 81402) Sep 12 23:13:33.940726 dv-cinder-852009-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 12 23:13:33.942642 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: spawned uWSGI worker 1 (pid: 81416, cores: 1) Sep 12 23:13:33.942642 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: spawned uWSGI worker 2 (pid: 81417, cores: 1) Sep 12 23:13:33.942805 dv-cinder-852009-7-iscsi devstack@keystone.service[81402]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 12 23:13:38.204764 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 23:13:38.204764 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: from cryptography.utils import int_from_bytes Sep 12 23:13:38.205505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 23:13:38.205505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: from cryptography.utils import int_from_bytes Sep 12 23:13:38.205627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 23:13:38.205627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: from cryptography.utils import int_from_bytes Sep 12 23:13:38.205627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 23:13:38.205627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: from cryptography.utils import int_from_bytes Sep 12 23:13:38.254977 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 12 23:13:38.255137 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 12 23:13:38.255229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 12 23:13:38.255319 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 12 23:13:38.255399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] command line args: [] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 12 23:13:38.255477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] command line args: [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 12 23:13:38.255608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 12 23:13:38.255707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 12 23:13:38.255787 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ================================================================================ {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 12 23:13:38.255865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ================================================================================ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 12 23:13:38.256025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] admin_token = **** {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.256106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] admin_token = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.256270 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] config_dir = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.256358 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] config_dir = [] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.256436 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.256509 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.256616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] config_source = [] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.256701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] config_source = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.256781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] debug = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.256859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] debug = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.256977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.257282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.257418 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.257517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.257612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.257707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.257820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.257918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.258025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.258124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.258233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] list_limit = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.258336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.258434 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] log_config_append = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.258529 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_config_append = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.258632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.258742 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.258846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] log_dir = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.258941 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_dir = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.259036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] log_file = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.259137 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_file = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.259241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.259322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.259440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.259515 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.259655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.259772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.259871 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.259954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.260028 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.260147 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.260223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.260358 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.260935 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.261008 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.261127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.261209 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.261294 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.261419 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.261496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.261620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.261697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.261819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.261928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.262030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.262173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.262263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.262415 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.262500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.262687 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.262777 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.263169 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.263249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.263332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] publish_errors = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.263404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] publish_errors = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.263595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.263692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.263814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.263900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.263979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.264050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.264219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.264359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.264445 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.264523 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.264664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] standard_threads = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.264745 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] standard_threads = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.264891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.265026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.265150 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.265270 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.265386 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.265499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.265499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] use_journal = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.265780 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_journal = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.265901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] use_json = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.266023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_json = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.266139 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] use_stderr = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.266253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_stderr = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.266399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] use_syslog = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.266530 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_syslog = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.266647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.266768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:13:38.266891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.267007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.267129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.267259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.267383 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.267499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.267636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.267765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.267883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.268010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.268132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.268254 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.268369 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.268490 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.268605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.268728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.268844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.268966 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.269088 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.269203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.269316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.269439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.269561 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.269677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.269790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.269926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.270043 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.270166 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.270289 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.270404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.270519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.270640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.270763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.270892 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.271009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.271131 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.271253 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.271368 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.271489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.271717 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.271838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.271960 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.272077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.272191 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.272314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.272430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.272545 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.272666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.272789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.272912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.273045 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.273170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.273285 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.273400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.273515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.273629 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.273752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.273869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.273984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.274105 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.274228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.274344 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.274470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.274586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.274700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.274821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.274948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.275071 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.275195 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.275312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.275427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.275541 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.275708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.275833 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.276540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.276672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.276788 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.276903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.277018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.277133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.277261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.277380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.277495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.277609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.277723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.277845 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.278036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.278154 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.278270 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.278386 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.278508 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.278624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.278747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.278864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.278980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.279094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.279209 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.279325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.279450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.279593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.279722 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.279839 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.279954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.280052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.280178 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.280383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.280512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.280691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.280795 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.280891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.280997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.281104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.281219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.281614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.281745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.281745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.281976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.281976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.282191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.282191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.282405 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.282528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.282645 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.282760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.282886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.282886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.283112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.283112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.283329 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.283457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.283623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.283742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.283858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.283983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.284100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.284223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.284339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.284455 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.284569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.284684 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.284798 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.284923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.285044 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.285159 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.285276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.285399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.285522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.285637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.285754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.285869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.285983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.286098 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.286219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.286343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.286466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.286588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.286705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.286820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.286935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.287058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.287175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.287289 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.287404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.287518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.287656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.287784 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.287903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.288025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.288141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.288255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.288370 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.288493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.288609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.288723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.288837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.288952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.289067 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.289190 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.289306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.289421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.289535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.289649 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.289769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.289959 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.290079 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.290194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.290315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.290431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.290555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.290680 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.290810 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.290927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.291043 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.291148 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.291261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.291375 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.291499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.291671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.291803 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.291921 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.292037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.292153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.292277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.292394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.292509 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.292623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.292738 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.292852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.292975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.293094 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.293209 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.293323 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.293447 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.293593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] auth.external = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.293720 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.293844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.293970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.external = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.294089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.294206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.294321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.294437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.294562 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] auth.password = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.294688 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.294812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] auth.token = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.294929 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.password = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.295049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.295167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.token = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.295282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.295406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.295524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.295655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.295772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.295888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.296004 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.296133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.296252 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.296369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.296484 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.296600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.296714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] credential.caching = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.296838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.296956 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.297078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.297195 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.297319 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.297436 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.297561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.297679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.297795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.297918 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.298034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.298156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.298282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.298400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.298517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.298639 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.298756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.298872 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.298997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.299114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.299231 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.299348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.299471 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.299602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.299733 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.299851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.299967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.300083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.300198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.300321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.300451 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.300569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] federation.caching = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.300685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.300801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.300917 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.301033 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.301158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.301283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.301400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.301517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.301633 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.301749 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.301874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.301991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.302107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.302223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.302339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.302463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.302587 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.302706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.302823 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.302946 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.303068 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.303197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.303315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.303432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity.caching = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.303548 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.303699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.303816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.304011 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.304132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.304248 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.304364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.304480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.304597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.304723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.304841 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.304957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.305075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.305191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.305307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.305431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.305549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.305666 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.305781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.305903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.306020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.306144 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.306261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.306377 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.306493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.306609 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.306726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.306850 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.306968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.307085 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.307201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.307316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.307433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.307561 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.307706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.307826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.307942 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.308058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.308173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.308298 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.308415 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.308538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.308655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.308771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.308886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.309010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.309128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.309244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.309350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.309463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.309579 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.309703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.309822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.309937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.310053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.310169 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.310284 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.310413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.310532 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.310648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.310765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.310881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.310997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.311124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.311242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.311364 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.311481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.311642 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.311774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.311892 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.312008 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.312124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.312240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.312355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.312479 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.312603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.312722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.312837 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.312953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.313070 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.313194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.313312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.313434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.313552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.313667 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.313783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.313920 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.314046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.314162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.314277 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.314394 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.314510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.314638 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.314757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.314874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.314990 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.315111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.315227 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.315351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.315469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.315615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.315735 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.315852 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.315968 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.316096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.316215 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.316332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.316448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.316564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.316680 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.316805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.316923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.317040 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.317156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.317272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.317387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.317511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.317628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.317745 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.317860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.317977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.318093 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.318218 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.318335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.318451 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.318566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.318681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.318796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.318920 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.319038 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.319154 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.319271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.319386 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.319502 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.319644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.319764 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.319880 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.319996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.320112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.320227 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.320352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.320469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.320469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.320687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.320802 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.320918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.321042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.321160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.321289 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.321407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.321407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.321623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.321623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.321851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.321967 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.322082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.322206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.322328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.322455 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.322573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.322688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.322804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.323443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.323562 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.323761 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.323881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.323997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.324126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.324247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.324363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.324488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.324606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.324722 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.324838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.324954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] resource.caching = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.325072 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.325197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.325316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.325431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.325547 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.325662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.325778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.325902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.326020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.326136 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.326252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.326368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.326484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.326609 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.326776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.326895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.327011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.327127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.327315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.327435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] role.caching = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.327551 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.327690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] role.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.327806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.327921 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.328050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] role.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.328169 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.328285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] role.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.328401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.328518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.328633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.328759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.328914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.329037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.329154 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.329270 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.329270 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.329497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.329628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.329746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.329862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.329978 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.329978 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.330204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.330322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.330437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.330437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.330654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.330770 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.330894 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.330894 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.331114 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.331114 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.331332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.331447 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.331611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.331736 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.331854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.331970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.332086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.332201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.332331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.332450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.332568 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.332568 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.332786 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.332786 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.333016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.333133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.333248 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.333248 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.333466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.333590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.333708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.333824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.333941 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.334057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.334173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.334311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.334430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.334546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.334662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.334777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.334893 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.335027 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.335145 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.335262 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.335377 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.335492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.335623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.335753 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] token.caching = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.335872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.335988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.336105 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.336222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.336338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.336462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.336581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.336698 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.336814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.336929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.337045 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.337171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.337302 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.337420 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.337536 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.337651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.337767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.337895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.338013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.338130 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.338247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.338363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.338478 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.339172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.339292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.339408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.339524 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.339658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.339775 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.339904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.340032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.340159 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.340278 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.340395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.340512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.340637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.connection = **** {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.340763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.340881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.341006 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.341124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.341240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.341357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.341473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.341589 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.341714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.341832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.341947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.342063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.342178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.342293 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.342419 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.342537 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.342665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.342782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.342898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.343015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.343139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.343258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.343374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.343490 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.343657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.343776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.343906 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.344024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.344147 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.344264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.344264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.344481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.344605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.344723 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.344838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.344954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.345080 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 12 23:13:38.345198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.345198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.345198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:13:38.345198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 12 23:13:38.369222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.369449 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.369692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.369782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.369951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.370179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.378508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.383558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.407800 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.408031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.412412 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.412644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.461492 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.461896 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.509526 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.509747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.509836 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.510002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.665095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.665281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.665421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.665502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.665580 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.665677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.665755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.665830 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.665932 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.666016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.666148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.666231 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.691069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.691353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.691454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.691636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.691744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.691862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.691947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.692080 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.692165 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.692296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.692381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.692511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.692593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.692725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.703863 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.704057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.717196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.717320 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.717484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.717591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.717713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.717798 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.717934 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.718017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.718151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.718242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.718365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.718447 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.718581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.718663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:13:38.736324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.736499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.736584 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 23:13:38.736707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 23:13:38.736867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.737048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.737143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 23:13:38.737248 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 23:13:38.737331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.737522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.737604 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.737785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.737868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.738045 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.738126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.738301 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.747680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.747849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.747963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.748129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.748238 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.748397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.748526 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.748678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.755478 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.755578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.758442 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.758540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.758731 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.758811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.759015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.759096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.759298 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.759387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.759591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.759674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.759869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.759950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.760139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.760221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.760414 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.760496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.760687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.760776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.760958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.761038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.761225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.761306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.784582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.784676 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.786032 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.786115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.786317 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.786449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.790978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.791069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.791266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.791350 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.791555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.792183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.798362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.798447 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.798650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.798732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.804230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.804361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.804521 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.804663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.804803 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.804958 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.805086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.805251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.805362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.805535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.805638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.805820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.805903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.806099 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.806179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.806376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.821582 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.821867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.821949 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.822186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.826259 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.826581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.829470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.829766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.829847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.830121 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.834015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.834334 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.837172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.837549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.840449 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.840752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.840867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.841038 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.841170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.841324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.841460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.841603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.841749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.841884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.842041 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.842321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.858558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.859621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.859725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.859795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.859858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.859919 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.859997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.859997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.866217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.866633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.867791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.868191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.869323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.869612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.869731 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.870068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.872383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.872672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.872811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.872956 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.873102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.873242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.873387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.873642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.873726 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.874042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.884794 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.885081 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.885219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.885899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.886034 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.886186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.886320 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.886454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.886586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.886814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.886901 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.887093 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.887178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.887378 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.902502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.902683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.905447 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.905604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.908626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.908734 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.910118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.910276 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.911631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.911760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.911921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.912031 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.912317 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.912469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.919218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.919312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.920692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.920797 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.923550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.923660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.923974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.924059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.924260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.924357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.924537 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.924628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.932871 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.932964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.933184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.933280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.933467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.933550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.940326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.940423 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:13:38.940613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.940696 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.940893 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.940975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.941164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.941254 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.941435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.941518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.941703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.941786 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.941982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.942075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.942257 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.942340 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.942531 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.942614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.942796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.942878 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.943073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.943195 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.943334 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.943415 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.943617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.943744 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.943939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.944076 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:13:38.978655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d2cd2979c0 pid: 81416 (default app) Sep 12 23:13:38.978855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d2cd2979c0 pid: 81417 (default app) Sep 12 23:13:38.981916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1878f23-ffff-4022-9e6e-2877a4b2f99d None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:38.982204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1878f23-ffff-4022-9e6e-2877a4b2f99d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:38.982476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1878f23-ffff-4022-9e6e-2877a4b2f99d None None] PATH_INFO: `/v3` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:38.986616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Mon Sep 12 23:13:38 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:13:45.737706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e050f85-6586-4589-ba3e-c16949646293 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:45.738215 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e050f85-6586-4589-ba3e-c16949646293 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:45.738317 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e050f85-6586-4589-ba3e-c16949646293 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:45.742474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:13:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:13:45.748233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc1ded-9236-4873-a186-adf5276bd8a0 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:45.748826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc1ded-9236-4873-a186-adf5276bd8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:45.748826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc1ded-9236-4873-a186-adf5276bd8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:45.819699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: INFO dbcounter [None req-ecfc1ded-9236-4873-a186-adf5276bd8a0 None None] Registered counter for database keystone Sep 12 23:13:45.839513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG oslo_db.sqlalchemy.engines [None req-ecfc1ded-9236-4873-a186-adf5276bd8a0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81416) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 12 23:13:45.845726 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writer thread running {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 12 23:13:46.021497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG passlib.handlers.bcrypt [None req-ecfc1ded-9236-4873-a186-adf5276bd8a0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81416) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 12 23:13:46.021803 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG passlib.handlers.bcrypt [None req-ecfc1ded-9236-4873-a186-adf5276bd8a0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81416) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 12 23:13:46.079817 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-ecfc1ded-9236-4873-a186-adf5276bd8a0 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:13:46.081419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ecfc1ded-9236-4873-a186-adf5276bd8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:13:46.206025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:13:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 12 23:13:46.696516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e050f85-6586-4589-ba3e-c16949646293 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:46.765967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: INFO dbcounter [None req-5e050f85-6586-4589-ba3e-c16949646293 None None] Registered counter for database keystone Sep 12 23:13:46.786967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG oslo_db.sqlalchemy.engines [None req-5e050f85-6586-4589-ba3e-c16949646293 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81417) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 12 23:13:46.793143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writer thread running {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 12 23:13:46.914858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e050f85-6586-4589-ba3e-c16949646293 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:46.919177 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258a8f9-7c84-4c67-9512-5fda3d2b58f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:46.919856 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258a8f9-7c84-4c67-9512-5fda3d2b58f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:46.920144 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258a8f9-7c84-4c67-9512-5fda3d2b58f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:46.920420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258a8f9-7c84-4c67-9512-5fda3d2b58f1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:46.928953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258a8f9-7c84-4c67-9512-5fda3d2b58f1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:46.981423 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d258a8f9-7c84-4c67-9512-5fda3d2b58f1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 12 23:13:46.981714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258a8f9-7c84-4c67-9512-5fda3d2b58f1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:46.991123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Mon Sep 12 23:13:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 23:13:46.997011 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc1ded-9236-4873-a186-adf5276bd8a0 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:47.013626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc1ded-9236-4873-a186-adf5276bd8a0 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:47.018113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff1968-a0ea-4ab8-bd11-d751e4a9fb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:47.019216 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff1968-a0ea-4ab8-bd11-d751e4a9fb59 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:47.019499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff1968-a0ea-4ab8-bd11-d751e4a9fb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:47.019803 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff1968-a0ea-4ab8-bd11-d751e4a9fb59 admin admin] PATH_INFO: `/v3/projects/5b515d316021473bab5135bffb78aaff` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:47.021397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff1968-a0ea-4ab8-bd11-d751e4a9fb59 admin admin] RBAC: Authorizing `identity:get_project(project_id=5b515d316021473bab5135bffb78aaff)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:47.074308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8fff1968-a0ea-4ab8-bd11-d751e4a9fb59 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 12 23:13:47.074654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff1968-a0ea-4ab8-bd11-d751e4a9fb59 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:47.079115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Mon Sep 12 23:13:46 2022] GET /identity/v3/projects/5b515d316021473bab5135bffb78aaff => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:13:49.192282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388b427-bf5e-450e-ae5c-80c386f6a238 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:49.192756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388b427-bf5e-450e-ae5c-80c386f6a238 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:49.192756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388b427-bf5e-450e-ae5c-80c386f6a238 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:49.196811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:13:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:13:49.202359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10853c85-c739-46b2-8e58-e0437370d41d None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:49.202790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10853c85-c739-46b2-8e58-e0437370d41d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:49.202927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10853c85-c739-46b2-8e58-e0437370d41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:49.223169 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-10853c85-c739-46b2-8e58-e0437370d41d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:13:49.224755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-10853c85-c739-46b2-8e58-e0437370d41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:13:49.233192 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:13:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 12 23:13:49.502883 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388b427-bf5e-450e-ae5c-80c386f6a238 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:49.514866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388b427-bf5e-450e-ae5c-80c386f6a238 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:49.519038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c1b3f-f925-423c-9378-bcabe3839fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:49.519650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c1b3f-f925-423c-9378-bcabe3839fde admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:49.519932 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c1b3f-f925-423c-9378-bcabe3839fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:49.520205 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c1b3f-f925-423c-9378-bcabe3839fde admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:49.527787 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c1b3f-f925-423c-9378-bcabe3839fde admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:49.529539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c1b3f-f925-423c-9378-bcabe3839fde admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:49.538310 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Mon Sep 12 23:13:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:13:49.544155 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10853c85-c739-46b2-8e58-e0437370d41d None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:49.553253 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10853c85-c739-46b2-8e58-e0437370d41d None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:49.558237 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dc428-b688-4c17-82a1-f634c32d3e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:49.558860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dc428-b688-4c17-82a1-f634c32d3e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:49.559147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dc428-b688-4c17-82a1-f634c32d3e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:49.559426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dc428-b688-4c17-82a1-f634c32d3e73 admin admin] PATH_INFO: `/v3/users/318df21351594db286347c68d09ec473` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:49.560780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720dc428-b688-4c17-82a1-f634c32d3e73 admin admin] RBAC: Authorizing `identity:get_user(user_id=318df21351594db286347c68d09ec473)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:49.562607 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720dc428-b688-4c17-82a1-f634c32d3e73 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:49.564166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Mon Sep 12 23:13:49 2022] GET /identity/v3/users/318df21351594db286347c68d09ec473 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:13:51.786899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0b615-7ec0-46bc-9366-3c6d8abd32f1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:51.787455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0b615-7ec0-46bc-9366-3c6d8abd32f1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:51.787455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0b615-7ec0-46bc-9366-3c6d8abd32f1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:51.788818 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:13:51.794474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a774-c251-42ad-bb19-2b688bcf903b None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:51.794854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a774-c251-42ad-bb19-2b688bcf903b None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:51.795148 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a774-c251-42ad-bb19-2b688bcf903b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:51.800000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca857df-7619-4ac4-83aa-46e2dee8d19f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:51.800287 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca857df-7619-4ac4-83aa-46e2dee8d19f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:51.800565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca857df-7619-4ac4-83aa-46e2dee8d19f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:51.801795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:13:51.806916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517606c-b7e6-4a2c-9f8d-9e9f6e348c45 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:51.807197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517606c-b7e6-4a2c-9f8d-9e9f6e348c45 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:51.808754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517606c-b7e6-4a2c-9f8d-9e9f6e348c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:51.814905 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-2b87a774-c251-42ad-bb19-2b688bcf903b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:13:51.816622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2b87a774-c251-42ad-bb19-2b688bcf903b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:13:51.848634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG passlib.handlers.bcrypt [None req-9517606c-b7e6-4a2c-9f8d-9e9f6e348c45 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81417) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 12 23:13:51.849012 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG passlib.handlers.bcrypt [None req-9517606c-b7e6-4a2c-9f8d-9e9f6e348c45 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81417) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 12 23:13:51.896793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 6/12] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:13:51.898840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9517606c-b7e6-4a2c-9f8d-9e9f6e348c45 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:13:51.900427 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9517606c-b7e6-4a2c-9f8d-9e9f6e348c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:13:51.998096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:13:52.160458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87a774-c251-42ad-bb19-2b688bcf903b None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:52.184263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:52.264842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517606c-b7e6-4a2c-9f8d-9e9f6e348c45 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:52.266202 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87a774-c251-42ad-bb19-2b688bcf903b None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:52.303456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431408d3-d5e0-42c0-b86e-11e78423c11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:52.304113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431408d3-d5e0-42c0-b86e-11e78423c11f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:52.304395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431408d3-d5e0-42c0-b86e-11e78423c11f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:52.304670 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431408d3-d5e0-42c0-b86e-11e78423c11f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:52.315996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431408d3-d5e0-42c0-b86e-11e78423c11f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-431408d3-d5e0-42c0-b86e-11e78423c11f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:13:52.324816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:13:52.325783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:13:52.326869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:13:52.326869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:13:52.326869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:13:52.326869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:13:52.326869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:13:52.326869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:13:52.326869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:13:52.326869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:13:52.326869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:13:52.326869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:13:52.326869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:13:52.326869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:13:52.326869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 7/14] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:13:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:13:52.331741 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431408d3-d5e0-42c0-b86e-11e78423c11f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:52.367136 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517606c-b7e6-4a2c-9f8d-9e9f6e348c45 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:52.370201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:52.407701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0f225-5b18-42e9-8556-603e469251df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:52.408558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0f225-5b18-42e9-8556-603e469251df None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:52.408840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0f225-5b18-42e9-8556-603e469251df None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:52.409115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0f225-5b18-42e9-8556-603e469251df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:52.413050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:52.431003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431408d3-d5e0-42c0-b86e-11e78423c11f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:52.463344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078893f0-2960-4f2f-943f-f14883577b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:52.464550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078893f0-2960-4f2f-943f-f14883577b73 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:52.464819 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078893f0-2960-4f2f-943f-f14883577b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:52.465089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078893f0-2960-4f2f-943f-f14883577b73 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:52.465856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078893f0-2960-4f2f-943f-f14883577b73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:13:52.466158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078893f0-2960-4f2f-943f-f14883577b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:52.467548 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078893f0-2960-4f2f-943f-f14883577b73 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:52.474290 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0f225-5b18-42e9-8556-603e469251df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:52.475824 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0f225-5b18-42e9-8556-603e469251df None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:52.478733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 8/15] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:13:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:13:52.484328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078893f0-2960-4f2f-943f-f14883577b73 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:52.570970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 8/16] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:13:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:13:52.577101 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0f225-5b18-42e9-8556-603e469251df None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:52.579753 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:52.587114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078893f0-2960-4f2f-943f-f14883577b73 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:52.620450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0374537-39c8-4490-aaff-6812e510894e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:52.621084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0374537-39c8-4490-aaff-6812e510894e None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:52.621362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0374537-39c8-4490-aaff-6812e510894e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:52.621633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0374537-39c8-4490-aaff-6812e510894e None admin] PATH_INFO: `/v3/users/318df21351594db286347c68d09ec473` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:52.622771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0374537-39c8-4490-aaff-6812e510894e None admin] RBAC: Authorizing `identity:get_user(user_id=318df21351594db286347c68d09ec473)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:52.624249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0374537-39c8-4490-aaff-6812e510894e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:52.625809 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 9/17] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:13:52 2022] GET /identity/v3/users/318df21351594db286347c68d09ec473 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:13:52.631113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0374537-39c8-4490-aaff-6812e510894e None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:52.645262 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:52.683625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0f225-5b18-42e9-8556-603e469251df None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:52.716752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689311f-a731-49e3-a89b-f7c822690d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:52.717595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689311f-a731-49e3-a89b-f7c822690d3b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:52.717874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689311f-a731-49e3-a89b-f7c822690d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:52.718146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689311f-a731-49e3-a89b-f7c822690d3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:52.727071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0374537-39c8-4490-aaff-6812e510894e None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:52.736397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689311f-a731-49e3-a89b-f7c822690d3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:52.737846 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689311f-a731-49e3-a89b-f7c822690d3b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e689311f-a731-49e3-a89b-f7c822690d3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:13:52.755445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:13:52.756382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:13:52.757428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:13:52.757428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:13:52.757428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:13:52.757428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 9/18] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:13:52.760768 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d07b7-e010-4a5e-83c9-a6ab9f938679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:52.761394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d07b7-e010-4a5e-83c9-a6ab9f938679 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:52.761679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d07b7-e010-4a5e-83c9-a6ab9f938679 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:52.761949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d07b7-e010-4a5e-83c9-a6ab9f938679 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:52.762367 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689311f-a731-49e3-a89b-f7c822690d3b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:52.763346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d07b7-e010-4a5e-83c9-a6ab9f938679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:52.764844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d07b7-e010-4a5e-83c9-a6ab9f938679 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:52.766359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 10/19] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:13:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:13:52.771800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d07b7-e010-4a5e-83c9-a6ab9f938679 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:52.820792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:52.852689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:52.866517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689311f-a731-49e3-a89b-f7c822690d3b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:52.871997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d07b7-e010-4a5e-83c9-a6ab9f938679 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:52.899762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b281671-81a6-4da6-a678-cc7e772deda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:52.900437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b281671-81a6-4da6-a678-cc7e772deda4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:52.900734 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b281671-81a6-4da6-a678-cc7e772deda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:52.901502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b281671-81a6-4da6-a678-cc7e772deda4 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:52.902145 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b281671-81a6-4da6-a678-cc7e772deda4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:13:52.902451 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b281671-81a6-4da6-a678-cc7e772deda4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:52.903846 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b281671-81a6-4da6-a678-cc7e772deda4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:52.905229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c98c7-5c87-436d-8152-82905eed6553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:52.905834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c98c7-5c87-436d-8152-82905eed6553 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:52.906111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c98c7-5c87-436d-8152-82905eed6553 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:52.906383 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c98c7-5c87-436d-8152-82905eed6553 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:52.907346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c98c7-5c87-436d-8152-82905eed6553 None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.domain.id=default, user.id=318df21351594db286347c68d09ec473) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:13:52.907678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c98c7-5c87-436d-8152-82905eed6553 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:52.909089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c98c7-5c87-436d-8152-82905eed6553 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:52.919587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 11/20] 10.222.0.27 () {56 vars in 1322 bytes} [Mon Sep 12 23:13:52 2022] GET /identity/v3/role_assignments?user.id=318df21351594db286347c68d09ec473&scope.domain.id=default&role.id=23538dc4c770454e80dd11ed8e498539 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:13:52.921847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 10/21] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:13:55.231908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de65b-c686-4a5f-92ba-ed09d116ccd1 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:55.232391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de65b-c686-4a5f-92ba-ed09d116ccd1 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:55.232391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de65b-c686-4a5f-92ba-ed09d116ccd1 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:55.233995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 12/22] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:13:55.245634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578d07a-fe89-409c-a86d-599581baa9e1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:55.246089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578d07a-fe89-409c-a86d-599581baa9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:55.246263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578d07a-fe89-409c-a86d-599581baa9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:55.277092 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3578d07a-fe89-409c-a86d-599581baa9e1 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:13:55.278302 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3578d07a-fe89-409c-a86d-599581baa9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:13:55.363550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 11/23] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:13:55.690496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d577d7-f5ab-446e-a37b-94d854b12a16 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:55.691054 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d577d7-f5ab-446e-a37b-94d854b12a16 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:55.691470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d577d7-f5ab-446e-a37b-94d854b12a16 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:55.693327 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 13/24] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:13:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:13:55.699524 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d0876-d3da-43d7-960e-96855288f430 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:55.700059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d0876-d3da-43d7-960e-96855288f430 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:55.700502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d0876-d3da-43d7-960e-96855288f430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:55.728596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8b0d0876-d3da-43d7-960e-96855288f430 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:13:55.730137 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8b0d0876-d3da-43d7-960e-96855288f430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:13:55.786681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:55.816152 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb6a24-bb61-4e2a-8bbc-e36336653339 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:55.817022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 12/25] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:13:55.817228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb6a24-bb61-4e2a-8bbc-e36336653339 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:55.817691 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb6a24-bb61-4e2a-8bbc-e36336653339 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:55.819191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 14/26] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:13:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:13:55.837714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a6658-bc32-4302-8222-e4115d76f8d4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:55.838009 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a6658-bc32-4302-8222-e4115d76f8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:55.838290 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a6658-bc32-4302-8222-e4115d76f8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:55.857339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb6a24-bb61-4e2a-8bbc-e36336653339 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:55.861835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-605a6658-bc32-4302-8222-e4115d76f8d4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:13:55.863052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-605a6658-bc32-4302-8222-e4115d76f8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:13:55.997222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 13/27] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:13:56.094614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb6a24-bb61-4e2a-8bbc-e36336653339 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:56.139794 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e51ad5-5031-4fbc-932e-da2bf8104312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:56.140745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e51ad5-5031-4fbc-932e-da2bf8104312 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:56.141244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e51ad5-5031-4fbc-932e-da2bf8104312 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:56.141723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e51ad5-5031-4fbc-932e-da2bf8104312 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:56.160760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e51ad5-5031-4fbc-932e-da2bf8104312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-15e51ad5-5031-4fbc-932e-da2bf8104312 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:13:56.166371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:13:56.167785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:13:56.168527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:13:56.168527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:13:56.168527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:13:56.168527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:13:56.168527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:13:56.168527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:13:56.168527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:13:56.168527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:13:56.168527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:13:56.168527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:13:56.168527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:13:56.168527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:13:56.170710 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 15/28] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:13:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:13:56.176553 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a6658-bc32-4302-8222-e4115d76f8d4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:56.273259 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e51ad5-5031-4fbc-932e-da2bf8104312 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:56.283714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:56.351623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a6658-bc32-4302-8222-e4115d76f8d4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:56.359904 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:56.421881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3458a4-c48a-4461-9d29-67f29e720d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:56.422563 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3458a4-c48a-4461-9d29-67f29e720d20 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:56.422850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3458a4-c48a-4461-9d29-67f29e720d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:56.423127 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3458a4-c48a-4461-9d29-67f29e720d20 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:56.423834 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3458a4-c48a-4461-9d29-67f29e720d20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:13:56.424142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3458a4-c48a-4461-9d29-67f29e720d20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:56.425666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3458a4-c48a-4461-9d29-67f29e720d20 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:56.450043 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 14/29] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:13:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:13:56.451759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80679d-1598-40cf-a3c2-a721ca0b672e None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:56.452223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80679d-1598-40cf-a3c2-a721ca0b672e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:56.452702 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80679d-1598-40cf-a3c2-a721ca0b672e None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:56.454397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 15/30] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:13:56.456054 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80679d-1598-40cf-a3c2-a721ca0b672e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:56.469626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e51ad5-5031-4fbc-932e-da2bf8104312 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:56.518826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c06d2-0358-4909-8887-54a64c2c40b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:56.519921 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c06d2-0358-4909-8887-54a64c2c40b6 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:56.520399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c06d2-0358-4909-8887-54a64c2c40b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:56.521001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c06d2-0358-4909-8887-54a64c2c40b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:56.535751 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c06d2-0358-4909-8887-54a64c2c40b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:56.538003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c06d2-0358-4909-8887-54a64c2c40b6 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-6c4c06d2-0358-4909-8887-54a64c2c40b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:13:56.557701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:13:56.558669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:13:56.559472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:13:56.559472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:13:56.559472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:13:56.559472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 16/31] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:13:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:13:56.559757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b60c4-c6a2-44d9-a5b1-782f1f727580 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:56.560271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b60c4-c6a2-44d9-a5b1-782f1f727580 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:56.560690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b60c4-c6a2-44d9-a5b1-782f1f727580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:56.580298 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-988b60c4-c6a2-44d9-a5b1-782f1f727580 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:13:56.581748 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-988b60c4-c6a2-44d9-a5b1-782f1f727580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:13:56.620531 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80679d-1598-40cf-a3c2-a721ca0b672e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:56.636241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:56.659714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 17/32] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:13:56.660748 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b60c4-c6a2-44d9-a5b1-782f1f727580 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:56.664427 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cee76-38f2-40b2-af91-f215c245c8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:56.665372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cee76-38f2-40b2-af91-f215c245c8ec None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:56.665908 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cee76-38f2-40b2-af91-f215c245c8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:56.666330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cee76-38f2-40b2-af91-f215c245c8ec None admin] PATH_INFO: `/v3/users/318df21351594db286347c68d09ec473` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:56.668157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cee76-38f2-40b2-af91-f215c245c8ec None admin] RBAC: Authorizing `identity:get_user(user_id=318df21351594db286347c68d09ec473)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:56.669980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cee76-38f2-40b2-af91-f215c245c8ec None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:56.672266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 16/33] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:13:56 2022] GET /identity/v3/users/318df21351594db286347c68d09ec473 => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:13:56.673309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cee76-38f2-40b2-af91-f215c245c8ec None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:56.750515 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:56.852285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cee76-38f2-40b2-af91-f215c245c8ec None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:56.866541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b60c4-c6a2-44d9-a5b1-782f1f727580 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:56.892963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55141956-f769-497b-8b38-5f009c2cc853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:56.893870 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55141956-f769-497b-8b38-5f009c2cc853 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:56.894316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55141956-f769-497b-8b38-5f009c2cc853 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:56.894788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55141956-f769-497b-8b38-5f009c2cc853 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:56.896157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55141956-f769-497b-8b38-5f009c2cc853 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:13:56.896939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55141956-f769-497b-8b38-5f009c2cc853 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:56.898809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55141956-f769-497b-8b38-5f009c2cc853 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:56.917819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 17/34] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:13:56.919527 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55141956-f769-497b-8b38-5f009c2cc853 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:56.924493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f34d24-6d80-450b-87a1-46a6ab78d793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:56.925131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f34d24-6d80-450b-87a1-46a6ab78d793 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:56.925414 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f34d24-6d80-450b-87a1-46a6ab78d793 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:56.925685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f34d24-6d80-450b-87a1-46a6ab78d793 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:56.926364 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f34d24-6d80-450b-87a1-46a6ab78d793 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:56.928158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f34d24-6d80-450b-87a1-46a6ab78d793 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:57.037153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:57.045112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55141956-f769-497b-8b38-5f009c2cc853 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:57.086870 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b41eb7-bd2b-456c-ae91-2fc1764885c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:57.087745 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b41eb7-bd2b-456c-ae91-2fc1764885c9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:57.088130 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b41eb7-bd2b-456c-ae91-2fc1764885c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:57.089130 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b41eb7-bd2b-456c-ae91-2fc1764885c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:57.090402 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b41eb7-bd2b-456c-ae91-2fc1764885c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:57.092000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b41eb7-bd2b-456c-ae91-2fc1764885c9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:57.093888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 18/35] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:13:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:13:57.094536 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b41eb7-bd2b-456c-ae91-2fc1764885c9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:57.102089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 18/36] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Sep 12 23:13:56 2022] POST /identity/v3/roles => generated 225 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:13:57.103427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f34d24-6d80-450b-87a1-46a6ab78d793 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:57.162061 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:57.216344 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b41eb7-bd2b-456c-ae91-2fc1764885c9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:57.226839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f34d24-6d80-450b-87a1-46a6ab78d793 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:57.254855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500091a-e6de-4b1a-85fc-1113620f22df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:57.255478 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500091a-e6de-4b1a-85fc-1113620f22df None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:57.255797 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500091a-e6de-4b1a-85fc-1113620f22df None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:57.256075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500091a-e6de-4b1a-85fc-1113620f22df None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:57.256741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500091a-e6de-4b1a-85fc-1113620f22df None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:57.258231 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500091a-e6de-4b1a-85fc-1113620f22df None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:57.268073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a39ae8-243f-4ec8-8784-e23d4810ac55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:57.269011 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a39ae8-243f-4ec8-8784-e23d4810ac55 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:57.270011 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a39ae8-243f-4ec8-8784-e23d4810ac55 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:57.270499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a39ae8-243f-4ec8-8784-e23d4810ac55 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:57.271913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a39ae8-243f-4ec8-8784-e23d4810ac55 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:57.273992 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a39ae8-243f-4ec8-8784-e23d4810ac55 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:57.292886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 19/37] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Sep 12 23:13:57 2022] POST /identity/v3/roles => generated 219 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:13:57.294034 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a39ae8-243f-4ec8-8784-e23d4810ac55 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:57.400127 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 19/38] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Sep 12 23:13:57 2022] POST /identity/v3/projects => generated 301 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:13:57.402296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110de46-76fc-45ff-a826-c4215dadfe10 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:57.402658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110de46-76fc-45ff-a826-c4215dadfe10 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:57.402945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110de46-76fc-45ff-a826-c4215dadfe10 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:57.404492 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 20/39] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:13:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:13:57.405626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6627e6-be48-4f08-9029-67c8baba9152 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:57.405920 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6627e6-be48-4f08-9029-67c8baba9152 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:57.406199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6627e6-be48-4f08-9029-67c8baba9152 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:57.409765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 21/40] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:13:57.410255 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada43e5-8dd9-461f-9ef5-71830d782afb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:57.410438 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada43e5-8dd9-461f-9ef5-71830d782afb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:57.410721 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada43e5-8dd9-461f-9ef5-71830d782afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:57.432688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a39ae8-243f-4ec8-8784-e23d4810ac55 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:57.441331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5ada43e5-8dd9-461f-9ef5-71830d782afb None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:13:57.443168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5ada43e5-8dd9-461f-9ef5-71830d782afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:13:57.448709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:57.458889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:57.492896 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4816a-1520-45a9-ba41-58e003f6854f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:57.493739 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4816a-1520-45a9-ba41-58e003f6854f None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:57.494133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4816a-1520-45a9-ba41-58e003f6854f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:57.494508 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4816a-1520-45a9-ba41-58e003f6854f None admin] PATH_INFO: `/v3/domains/default/users/318df21351594db286347c68d09ec473/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:57.497123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4816a-1520-45a9-ba41-58e003f6854f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=318df21351594db286347c68d09ec473, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:57.499025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4816a-1520-45a9-ba41-58e003f6854f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:57.519498 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 20/41] 10.222.0.27 () {58 vars in 1361 bytes} [Mon Sep 12 23:13:57 2022] PUT /identity/v3/domains/default/users/318df21351594db286347c68d09ec473/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:13:57.527835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1338d4-e755-41ac-9e34-950e377a5f6a None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:57.527835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1338d4-e755-41ac-9e34-950e377a5f6a None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:57.527835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1338d4-e755-41ac-9e34-950e377a5f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:57.547303 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-0c1338d4-e755-41ac-9e34-950e377a5f6a None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:13:57.547748 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 22/42] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:13:57.549404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-0c1338d4-e755-41ac-9e34-950e377a5f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:13:57.631205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 21/43] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:13:57.836826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada43e5-8dd9-461f-9ef5-71830d782afb None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:57.908340 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1338d4-e755-41ac-9e34-950e377a5f6a None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:57.945382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada43e5-8dd9-461f-9ef5-71830d782afb None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:57.980049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb4970-314f-4e02-8926-214b500ab394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:57.980917 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb4970-314f-4e02-8926-214b500ab394 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:57.981184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb4970-314f-4e02-8926-214b500ab394 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:57.981512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb4970-314f-4e02-8926-214b500ab394 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:57.982210 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb4970-314f-4e02-8926-214b500ab394 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:57.983768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb4970-314f-4e02-8926-214b500ab394 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:58.007611 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 23/44] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Sep 12 23:13:57 2022] POST /identity/v3/roles => generated 223 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:13:58.012953 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:58.021003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1338d4-e755-41ac-9e34-950e377a5f6a None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:58.063477 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ee967-1e95-46c9-b728-22281f9c0be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:58.064183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ee967-1e95-46c9-b728-22281f9c0be9 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:58.064469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ee967-1e95-46c9-b728-22281f9c0be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:58.064744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ee967-1e95-46c9-b728-22281f9c0be9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:58.066040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ee967-1e95-46c9-b728-22281f9c0be9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:58.067678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ee967-1e95-46c9-b728-22281f9c0be9 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:58.074004 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 22/45] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:13:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:13:58.075177 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb4970-314f-4e02-8926-214b500ab394 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:13:58.085187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:58.189280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb4970-314f-4e02-8926-214b500ab394 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:13:58.223378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2444c1d2-a90d-4bb7-97bc-ae4a1f37144b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:13:58.224027 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444c1d2-a90d-4bb7-97bc-ae4a1f37144b None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:58.224314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444c1d2-a90d-4bb7-97bc-ae4a1f37144b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:58.224593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444c1d2-a90d-4bb7-97bc-ae4a1f37144b None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:58.225260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444c1d2-a90d-4bb7-97bc-ae4a1f37144b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:13:58.226837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444c1d2-a90d-4bb7-97bc-ae4a1f37144b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:13:58.256912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 24/46] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Sep 12 23:13:58 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:13:59.714566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ced3e-b42c-4266-86ea-8ee57db67f53 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:59.715081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ced3e-b42c-4266-86ea-8ee57db67f53 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:59.715191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ced3e-b42c-4266-86ea-8ee57db67f53 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:59.716509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 23/47] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:13:59.721774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb219f6-22f3-46b7-ae48-77d5e709708f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:13:59.722150 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb219f6-22f3-46b7-ae48-77d5e709708f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:13:59.722454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb219f6-22f3-46b7-ae48-77d5e709708f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:13:59.740278 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2eb219f6-22f3-46b7-ae48-77d5e709708f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:13:59.741466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2eb219f6-22f3-46b7-ae48-77d5e709708f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:13:59.794029 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:13:59.808551 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 25/48] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:00.081848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ced3e-b42c-4266-86ea-8ee57db67f53 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:00.287516 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ced3e-b42c-4266-86ea-8ee57db67f53 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:00.323468 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13eec49-0130-464e-8373-327127754404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:00.324125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13eec49-0130-464e-8373-327127754404 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:00.324408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13eec49-0130-464e-8373-327127754404 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:00.324693 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13eec49-0130-464e-8373-327127754404 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:00.334490 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13eec49-0130-464e-8373-327127754404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-d13eec49-0130-464e-8373-327127754404 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:00.339332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:00.340188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:00.340936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:00.340936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:00.340936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:00.340936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:00.340936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:00.340936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:00.340936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:00.340936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:00.340936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:00.340936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:00.340936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:00.340936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:00.340936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 24/49] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:14:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:00.346166 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb219f6-22f3-46b7-ae48-77d5e709708f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:00.397362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f5e2e-49a7-43ca-bf3b-ed5594ccb765 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:00.397647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f5e2e-49a7-43ca-bf3b-ed5594ccb765 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:00.397929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f5e2e-49a7-43ca-bf3b-ed5594ccb765 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:00.399273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 25/50] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:14:00.404366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec020b-0103-4041-b8fb-e7892e483b1c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:00.404653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec020b-0103-4041-b8fb-e7892e483b1c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:00.404933 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec020b-0103-4041-b8fb-e7892e483b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:00.422608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-d0ec020b-0103-4041-b8fb-e7892e483b1c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:00.423848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d0ec020b-0103-4041-b8fb-e7892e483b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:00.441107 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:00.443973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb219f6-22f3-46b7-ae48-77d5e709708f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:00.478185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a555b-93a0-4c06-8c57-5fbfa8dc2180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:00.478799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a555b-93a0-4c06-8c57-5fbfa8dc2180 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:00.479111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a555b-93a0-4c06-8c57-5fbfa8dc2180 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:00.479390 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a555b-93a0-4c06-8c57-5fbfa8dc2180 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:00.480085 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a555b-93a0-4c06-8c57-5fbfa8dc2180 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:00.480397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a555b-93a0-4c06-8c57-5fbfa8dc2180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:00.481780 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a555b-93a0-4c06-8c57-5fbfa8dc2180 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:00.489590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 26/51] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:00.490474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 26/52] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:00.495722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a555b-93a0-4c06-8c57-5fbfa8dc2180 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:00.648361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:00.696704 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a555b-93a0-4c06-8c57-5fbfa8dc2180 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:00.728021 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9737d64-5bfb-4b22-aa51-84ca767b62d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:00.728651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9737d64-5bfb-4b22-aa51-84ca767b62d4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:00.729465 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9737d64-5bfb-4b22-aa51-84ca767b62d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:00.729752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9737d64-5bfb-4b22-aa51-84ca767b62d4 None admin] PATH_INFO: `/v3/users/318df21351594db286347c68d09ec473` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:00.730870 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9737d64-5bfb-4b22-aa51-84ca767b62d4 None admin] RBAC: Authorizing `identity:get_user(user_id=318df21351594db286347c68d09ec473)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:00.732332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9737d64-5bfb-4b22-aa51-84ca767b62d4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:00.733904 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 27/53] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:00 2022] GET /identity/v3/users/318df21351594db286347c68d09ec473 => generated 247 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:00.739343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec020b-0103-4041-b8fb-e7892e483b1c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:00.765056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9737d64-5bfb-4b22-aa51-84ca767b62d4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:00.835886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec020b-0103-4041-b8fb-e7892e483b1c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:00.861894 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9737d64-5bfb-4b22-aa51-84ca767b62d4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:00.868600 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f246a-bfd8-4a3b-b21a-207588b6e9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:00.869236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f246a-bfd8-4a3b-b21a-207588b6e9ba None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:00.869539 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f246a-bfd8-4a3b-b21a-207588b6e9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:00.869905 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f246a-bfd8-4a3b-b21a-207588b6e9ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:00.871011 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f246a-bfd8-4a3b-b21a-207588b6e9ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:00.872518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f246a-bfd8-4a3b-b21a-207588b6e9ba None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:00.874086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 27/54] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:14:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:00.878659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:00.879880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f246a-bfd8-4a3b-b21a-207588b6e9ba None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:00.895949 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052379b-d6da-4aa7-9a90-48c5240cb38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:00.896579 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052379b-d6da-4aa7-9a90-48c5240cb38a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:00.896864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052379b-d6da-4aa7-9a90-48c5240cb38a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:00.897144 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052379b-d6da-4aa7-9a90-48c5240cb38a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:00.898262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052379b-d6da-4aa7-9a90-48c5240cb38a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:00.899799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052379b-d6da-4aa7-9a90-48c5240cb38a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:00.901373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 28/55] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:14:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:00.903482 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:00.907260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052379b-d6da-4aa7-9a90-48c5240cb38a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:00.981702 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f246a-bfd8-4a3b-b21a-207588b6e9ba None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:01.006897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052379b-d6da-4aa7-9a90-48c5240cb38a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:01.015765 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad157f6-2d82-4a08-9963-bccdead06a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:01.016373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad157f6-2d82-4a08-9963-bccdead06a8f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:01.016655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad157f6-2d82-4a08-9963-bccdead06a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:01.016932 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad157f6-2d82-4a08-9963-bccdead06a8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:01.017656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad157f6-2d82-4a08-9963-bccdead06a8f None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.domain.id=default, user.id=318df21351594db286347c68d09ec473) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:01.017957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad157f6-2d82-4a08-9963-bccdead06a8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:01.019399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad157f6-2d82-4a08-9963-bccdead06a8f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:01.024957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 28/56] 10.222.0.27 () {56 vars in 1322 bytes} [Mon Sep 12 23:14:00 2022] GET /identity/v3/role_assignments?user.id=318df21351594db286347c68d09ec473&scope.domain.id=default&role.id=23538dc4c770454e80dd11ed8e498539 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:01.040893 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fb472-9f2b-4129-8f4d-c92436717f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:01.041527 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb472-9f2b-4129-8f4d-c92436717f96 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:01.041808 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb472-9f2b-4129-8f4d-c92436717f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:01.042082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb472-9f2b-4129-8f4d-c92436717f96 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:01.042756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fb472-9f2b-4129-8f4d-c92436717f96 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:01.044282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fb472-9f2b-4129-8f4d-c92436717f96 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:01.074093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 29/57] 10.222.0.27 () {60 vars in 1140 bytes} [Mon Sep 12 23:14:00 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:14:03.240135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd3618-483e-46fd-8dfe-b8b1f0956a41 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:03.240618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd3618-483e-46fd-8dfe-b8b1f0956a41 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:03.240618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd3618-483e-46fd-8dfe-b8b1f0956a41 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:03.241947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 29/58] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:03.247651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57c545-87e8-4ceb-8887-66f0e8060c3a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:03.248075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57c545-87e8-4ceb-8887-66f0e8060c3a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:03.248219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57c545-87e8-4ceb-8887-66f0e8060c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:03.266412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8a57c545-87e8-4ceb-8887-66f0e8060c3a None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:03.267638 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8a57c545-87e8-4ceb-8887-66f0e8060c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:03.296249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:03.332989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 30/59] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:03.609756 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd3618-483e-46fd-8dfe-b8b1f0956a41 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:03.692234 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:03.708156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd3618-483e-46fd-8dfe-b8b1f0956a41 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:03.741210 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5812a03-8a82-46a9-a8e6-fbdee10c32f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:03.741832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5812a03-8a82-46a9-a8e6-fbdee10c32f5 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:03.742115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5812a03-8a82-46a9-a8e6-fbdee10c32f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:03.742390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5812a03-8a82-46a9-a8e6-fbdee10c32f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:03.743487 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5812a03-8a82-46a9-a8e6-fbdee10c32f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:03.745553 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5812a03-8a82-46a9-a8e6-fbdee10c32f5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:03.747114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 30/60] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:14:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:03.752634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57c545-87e8-4ceb-8887-66f0e8060c3a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:03.845952 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57c545-87e8-4ceb-8887-66f0e8060c3a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:03.878820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccbf0a-313c-4d42-83a4-2a0a65d1e783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:03.879367 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccbf0a-313c-4d42-83a4-2a0a65d1e783 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:03.879677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccbf0a-313c-4d42-83a4-2a0a65d1e783 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:03.879956 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccbf0a-313c-4d42-83a4-2a0a65d1e783 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:03.880599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccbf0a-313c-4d42-83a4-2a0a65d1e783 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:03.882046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccbf0a-313c-4d42-83a4-2a0a65d1e783 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:03.910112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 31/61] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:14:03 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:14:07.244372 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11ea20-c0a5-415e-9c45-63cc46e0685e None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:07.245400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11ea20-c0a5-415e-9c45-63cc46e0685e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:07.245475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11ea20-c0a5-415e-9c45-63cc46e0685e None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:07.246777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 31/62] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:07.261682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5810eb-a2da-450e-a07d-b3946e346437 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:07.262138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5810eb-a2da-450e-a07d-b3946e346437 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:07.262234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5810eb-a2da-450e-a07d-b3946e346437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:07.288592 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2f5810eb-a2da-450e-a07d-b3946e346437 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:07.289812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2f5810eb-a2da-450e-a07d-b3946e346437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:07.310720 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882c1bc-9742-4993-9e24-8b33879deb21 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:07.311022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882c1bc-9742-4993-9e24-8b33879deb21 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:07.311279 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882c1bc-9742-4993-9e24-8b33879deb21 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:07.313219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:07.313864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 32/63] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:07.320394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a0e41-abb0-4c60-9b96-d8d821a524a1 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:07.320685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a0e41-abb0-4c60-9b96-d8d821a524a1 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:07.320993 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a0e41-abb0-4c60-9b96-d8d821a524a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:07.363709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-db5a0e41-abb0-4c60-9b96-d8d821a524a1 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:07.364935 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-db5a0e41-abb0-4c60-9b96-d8d821a524a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:07.413826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 32/64] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:07.432484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69ced-31fe-4f77-8240-9b5f0e1e0941 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:07.432779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69ced-31fe-4f77-8240-9b5f0e1e0941 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:07.433046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69ced-31fe-4f77-8240-9b5f0e1e0941 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:07.434378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 33/65] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:14:07.439489 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3cd9b4-e24e-4b79-97da-1842f2b24d1d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:07.439800 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3cd9b4-e24e-4b79-97da-1842f2b24d1d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:07.440080 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3cd9b4-e24e-4b79-97da-1842f2b24d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:07.467601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-da3cd9b4-e24e-4b79-97da-1842f2b24d1d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:07.468804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-da3cd9b4-e24e-4b79-97da-1842f2b24d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:07.487622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 33/66] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:07.558564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 34/67] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:07.757224 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae619-6898-4072-9bce-1501be414b67 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:07.757737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae619-6898-4072-9bce-1501be414b67 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:07.758160 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae619-6898-4072-9bce-1501be414b67 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:07.759791 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 34/68] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:07.765170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56d668-96a1-4887-bea2-15114c2f85e7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:07.765620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56d668-96a1-4887-bea2-15114c2f85e7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:07.766097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56d668-96a1-4887-bea2-15114c2f85e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:07.787402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae619-6898-4072-9bce-1501be414b67 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:07.799448 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5e56d668-96a1-4887-bea2-15114c2f85e7 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:07.802069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5e56d668-96a1-4887-bea2-15114c2f85e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:07.899685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 35/69] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:07.907615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56d668-96a1-4887-bea2-15114c2f85e7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:07.908223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:07.933497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:07.935727 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae619-6898-4072-9bce-1501be414b67 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:07.976290 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b16a4f-a383-4260-8562-e78846a1adf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:07.977155 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b16a4f-a383-4260-8562-e78846a1adf8 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:07.977623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b16a4f-a383-4260-8562-e78846a1adf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:07.977905 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b16a4f-a383-4260-8562-e78846a1adf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:07.989726 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b16a4f-a383-4260-8562-e78846a1adf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-16b16a4f-a383-4260-8562-e78846a1adf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:07.996254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:07.997119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:07.997887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:07.997887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:07.997887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:07.997887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:07.997887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:07.997887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:07.997887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:07.997887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:07.997887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:07.997887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:07.997887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:07.997887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:07.997887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 35/70] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:14:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:14:07.999547 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b16a4f-a383-4260-8562-e78846a1adf8 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:08.029984 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56d668-96a1-4887-bea2-15114c2f85e7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:08.066062 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98ae7d-080d-4cc2-91bf-d23e09aeca69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:08.066719 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98ae7d-080d-4cc2-91bf-d23e09aeca69 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:08.067003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98ae7d-080d-4cc2-91bf-d23e09aeca69 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:08.067277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98ae7d-080d-4cc2-91bf-d23e09aeca69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:08.079132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98ae7d-080d-4cc2-91bf-d23e09aeca69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-0e98ae7d-080d-4cc2-91bf-d23e09aeca69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:08.085666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:08.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:08.087276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:08.087276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:08.087276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:08.087276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:08.087276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:08.087276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:08.087276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:08.087276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:08.087276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:08.087276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:08.087276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:08.087276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:08.087276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 36/71] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:08.087931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419b7c3-d13d-4561-8bae-c54c95c63c1c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:08.088660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419b7c3-d13d-4561-8bae-c54c95c63c1c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:08.088660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419b7c3-d13d-4561-8bae-c54c95c63c1c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:08.094125 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 37/72] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:08.094125 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419b7c3-d13d-4561-8bae-c54c95c63c1c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:08.106423 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b16a4f-a383-4260-8562-e78846a1adf8 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:08.143561 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f86a7c-fca5-41bf-98a1-8b06abf57c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:08.144220 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f86a7c-fca5-41bf-98a1-8b06abf57c82 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:08.144507 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f86a7c-fca5-41bf-98a1-8b06abf57c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:08.144788 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f86a7c-fca5-41bf-98a1-8b06abf57c82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:08.155665 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f86a7c-fca5-41bf-98a1-8b06abf57c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-55f86a7c-fca5-41bf-98a1-8b06abf57c82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:08.160719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:08.161558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:08.162336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:08.162336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:08.162336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:08.162336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:08.162336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:08.162336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:08.162336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:08.162336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:08.162336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:08.162336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:08.162336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:08.162336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:08.162336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 36/73] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:14:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:08.162963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f86a7c-fca5-41bf-98a1-8b06abf57c82 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:08.168458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:08.181134 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:08.208805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419b7c3-d13d-4561-8bae-c54c95c63c1c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:08.242013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf93207-bb67-42d5-a393-6a0d8edc28df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:08.242622 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf93207-bb67-42d5-a393-6a0d8edc28df None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:08.242901 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf93207-bb67-42d5-a393-6a0d8edc28df None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:08.243172 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf93207-bb67-42d5-a393-6a0d8edc28df None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:08.243855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf93207-bb67-42d5-a393-6a0d8edc28df None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:08.244158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf93207-bb67-42d5-a393-6a0d8edc28df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:08.245578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf93207-bb67-42d5-a393-6a0d8edc28df None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:08.254774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 38/74] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:08.255942 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaee53-156a-4217-bfe2-4f2677604fdd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:08.256224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaee53-156a-4217-bfe2-4f2677604fdd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:08.256493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaee53-156a-4217-bfe2-4f2677604fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:08.268806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f86a7c-fca5-41bf-98a1-8b06abf57c82 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:08.278704 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8aeaee53-156a-4217-bfe2-4f2677604fdd None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:08.279935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8aeaee53-156a-4217-bfe2-4f2677604fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:08.302511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabd12a-fd7d-4752-a931-9928cb3f6234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:08.303114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabd12a-fd7d-4752-a931-9928cb3f6234 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:08.303399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabd12a-fd7d-4752-a931-9928cb3f6234 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:08.303700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabd12a-fd7d-4752-a931-9928cb3f6234 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:08.304358 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabd12a-fd7d-4752-a931-9928cb3f6234 None admin] RBAC: Adding query filter params (name=member) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:08.304663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabd12a-fd7d-4752-a931-9928cb3f6234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:08.306047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabd12a-fd7d-4752-a931-9928cb3f6234 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:08.313950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 37/75] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:08.314870 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabd12a-fd7d-4752-a931-9928cb3f6234 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:08.345123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 39/76] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:08.346082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeaee53-156a-4217-bfe2-4f2677604fdd None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:08.383353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:08.416207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:08.419232 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabd12a-fd7d-4752-a931-9928cb3f6234 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:08.454331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeaee53-156a-4217-bfe2-4f2677604fdd None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:08.454807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9e6d7-e0df-4b8a-83b4-3a8e60156417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:08.455420 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9e6d7-e0df-4b8a-83b4-3a8e60156417 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:08.455739 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9e6d7-e0df-4b8a-83b4-3a8e60156417 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:08.456025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9e6d7-e0df-4b8a-83b4-3a8e60156417 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:08.456690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9e6d7-e0df-4b8a-83b4-3a8e60156417 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:08.457666 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9e6d7-e0df-4b8a-83b4-3a8e60156417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:08.458393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9e6d7-e0df-4b8a-83b4-3a8e60156417 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:08.467704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 38/77] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:14:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:08.468908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9e6d7-e0df-4b8a-83b4-3a8e60156417 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:08.489534 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a6fcc-e1cc-4a43-baa1-913f3fe32d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:08.490167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a6fcc-e1cc-4a43-baa1-913f3fe32d0d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:08.490456 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a6fcc-e1cc-4a43-baa1-913f3fe32d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:08.490733 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a6fcc-e1cc-4a43-baa1-913f3fe32d0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:08.501577 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a6fcc-e1cc-4a43-baa1-913f3fe32d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-585a6fcc-e1cc-4a43-baa1-913f3fe32d0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:08.506554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:08.507416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:08.508434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:08.508434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:08.508434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:08.508434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:08.508434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:08.508434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:08.508434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:08.508434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:08.508434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:08.508434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:08.508434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:08.508434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:08.508434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 40/78] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:08.508434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a6fcc-e1cc-4a43-baa1-913f3fe32d0d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:08.586651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9e6d7-e0df-4b8a-83b4-3a8e60156417 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:08.613243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:08.615298 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a6fcc-e1cc-4a43-baa1-913f3fe32d0d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:08.623817 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d98fd2-a4e2-4eeb-b837-85b5b6f199b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:08.624453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d98fd2-a4e2-4eeb-b837-85b5b6f199b1 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:08.624743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d98fd2-a4e2-4eeb-b837-85b5b6f199b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:08.625049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d98fd2-a4e2-4eeb-b837-85b5b6f199b1 None admin] PATH_INFO: `/v3/users/318df21351594db286347c68d09ec473` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:08.626197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d98fd2-a4e2-4eeb-b837-85b5b6f199b1 None admin] RBAC: Authorizing `identity:get_user(user_id=318df21351594db286347c68d09ec473)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:08.627688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d98fd2-a4e2-4eeb-b837-85b5b6f199b1 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:08.629275 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 39/79] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:08 2022] GET /identity/v3/users/318df21351594db286347c68d09ec473 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:08.630041 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d98fd2-a4e2-4eeb-b837-85b5b6f199b1 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:08.642628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:08.652134 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70526562-2497-4ad0-859e-1fd0d2212c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:08.652765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70526562-2497-4ad0-859e-1fd0d2212c21 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:08.653050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70526562-2497-4ad0-859e-1fd0d2212c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:08.653337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70526562-2497-4ad0-859e-1fd0d2212c21 None admin] PATH_INFO: `/v3/users/28ab6d08782b4c26b0afe016bbcd8688` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:08.669340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70526562-2497-4ad0-859e-1fd0d2212c21 None admin] RBAC: Authorizing `identity:get_user(user_id=28ab6d08782b4c26b0afe016bbcd8688)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:08.670707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70526562-2497-4ad0-859e-1fd0d2212c21 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:08.672501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 41/80] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:08 2022] GET /identity/v3/users/28ab6d08782b4c26b0afe016bbcd8688 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:08.673111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70526562-2497-4ad0-859e-1fd0d2212c21 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:08.732443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d98fd2-a4e2-4eeb-b837-85b5b6f199b1 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:08.764999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12014b-134d-424a-a6ad-265b85ce24d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:08.765629 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12014b-134d-424a-a6ad-265b85ce24d4 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:08.765916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12014b-134d-424a-a6ad-265b85ce24d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:08.766205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12014b-134d-424a-a6ad-265b85ce24d4 None admin] PATH_INFO: `/v3/users/28ab6d08782b4c26b0afe016bbcd8688` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:08.767350 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12014b-134d-424a-a6ad-265b85ce24d4 None admin] RBAC: Authorizing `identity:get_user(user_id=28ab6d08782b4c26b0afe016bbcd8688)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:08.768841 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70526562-2497-4ad0-859e-1fd0d2212c21 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:08.768965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12014b-134d-424a-a6ad-265b85ce24d4 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:08.770419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 40/81] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:08 2022] GET /identity/v3/users/28ab6d08782b4c26b0afe016bbcd8688 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:08.771183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12014b-134d-424a-a6ad-265b85ce24d4 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:08.806127 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f93cc-12ee-4661-90f7-9e9e1663cc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:08.806737 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f93cc-12ee-4661-90f7-9e9e1663cc58 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:08.807080 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f93cc-12ee-4661-90f7-9e9e1663cc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:08.807358 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f93cc-12ee-4661-90f7-9e9e1663cc58 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:08.808053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f93cc-12ee-4661-90f7-9e9e1663cc58 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:08.808361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f93cc-12ee-4661-90f7-9e9e1663cc58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:08.809737 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f93cc-12ee-4661-90f7-9e9e1663cc58 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:08.817940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 42/82] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:08.818623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f93cc-12ee-4661-90f7-9e9e1663cc58 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:08.834903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:08.867961 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:08.873346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12014b-134d-424a-a6ad-265b85ce24d4 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:08.909963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855960b2-a283-4b1b-9396-332bf819c995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:08.910605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855960b2-a283-4b1b-9396-332bf819c995 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:08.910893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855960b2-a283-4b1b-9396-332bf819c995 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:08.911175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855960b2-a283-4b1b-9396-332bf819c995 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:08.912362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855960b2-a283-4b1b-9396-332bf819c995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:08.915906 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855960b2-a283-4b1b-9396-332bf819c995 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:08.915906 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 41/83] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:14:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:08.916220 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855960b2-a283-4b1b-9396-332bf819c995 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:08.926736 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f93cc-12ee-4661-90f7-9e9e1663cc58 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:08.960388 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefc1f8-e197-4437-b9c6-426d01bfde8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:08.961005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefc1f8-e197-4437-b9c6-426d01bfde8c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:08.961292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefc1f8-e197-4437-b9c6-426d01bfde8c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:08.961572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefc1f8-e197-4437-b9c6-426d01bfde8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:09.024953 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855960b2-a283-4b1b-9396-332bf819c995 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:09.028612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefc1f8-e197-4437-b9c6-426d01bfde8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:09.030151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefc1f8-e197-4437-b9c6-426d01bfde8c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:09.053508 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:09.058870 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c48dc-9db0-4930-a441-704295925169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:09.059557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c48dc-9db0-4930-a441-704295925169 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:09.059958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c48dc-9db0-4930-a441-704295925169 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:09.060303 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c48dc-9db0-4930-a441-704295925169 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:09.076668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:09.127902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c48dc-9db0-4930-a441-704295925169 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:09.129362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c48dc-9db0-4930-a441-704295925169 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:09.130091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 43/84] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:09.130931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefc1f8-e197-4437-b9c6-426d01bfde8c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:09.229039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefc1f8-e197-4437-b9c6-426d01bfde8c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:09.230179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 42/85] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:09.231024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c48dc-9db0-4930-a441-704295925169 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:09.260959 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:09.268086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff23f6-80ef-4085-94a8-7a3a37667136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:09.268704 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff23f6-80ef-4085-94a8-7a3a37667136 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:09.268988 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff23f6-80ef-4085-94a8-7a3a37667136 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:09.269268 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff23f6-80ef-4085-94a8-7a3a37667136 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:09.287240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:09.340775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c48dc-9db0-4930-a441-704295925169 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:09.357000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff23f6-80ef-4085-94a8-7a3a37667136 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:09.358459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff23f6-80ef-4085-94a8-7a3a37667136 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:09.376191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2bfa7-adfc-4f65-a40c-7d0726f0b60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:09.376823 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2bfa7-adfc-4f65-a40c-7d0726f0b60b None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:09.377105 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2bfa7-adfc-4f65-a40c-7d0726f0b60b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:09.377381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2bfa7-adfc-4f65-a40c-7d0726f0b60b None admin] PATH_INFO: `/v3/users/28ab6d08782b4c26b0afe016bbcd8688` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:09.378504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2bfa7-adfc-4f65-a40c-7d0726f0b60b None admin] RBAC: Authorizing `identity:get_user(user_id=28ab6d08782b4c26b0afe016bbcd8688)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:09.380007 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2bfa7-adfc-4f65-a40c-7d0726f0b60b None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:09.381588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 43/86] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:09 2022] GET /identity/v3/users/28ab6d08782b4c26b0afe016bbcd8688 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:09.382327 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2bfa7-adfc-4f65-a40c-7d0726f0b60b None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:09.455356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 44/87] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:09.456309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff23f6-80ef-4085-94a8-7a3a37667136 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:09.478065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:09.481707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2bfa7-adfc-4f65-a40c-7d0726f0b60b None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:09.502802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:09.516337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b42fb4-3c65-48e1-a976-a080d9b4d684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:09.516946 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b42fb4-3c65-48e1-a976-a080d9b4d684 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:09.517227 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b42fb4-3c65-48e1-a976-a080d9b4d684 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:09.517505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b42fb4-3c65-48e1-a976-a080d9b4d684 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:09.518162 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b42fb4-3c65-48e1-a976-a080d9b4d684 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:09.519620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b42fb4-3c65-48e1-a976-a080d9b4d684 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:09.559600 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 44/88] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:14:09 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:14:09.559600 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b42fb4-3c65-48e1-a976-a080d9b4d684 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:09.559874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff23f6-80ef-4085-94a8-7a3a37667136 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:09.606358 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3120d-6c1d-4869-9f57-5d37bd583bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:09.607021 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3120d-6c1d-4869-9f57-5d37bd583bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:09.607316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3120d-6c1d-4869-9f57-5d37bd583bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:09.607616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3120d-6c1d-4869-9f57-5d37bd583bf0 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:09.608828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3120d-6c1d-4869-9f57-5d37bd583bf0 None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:09.610341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3120d-6c1d-4869-9f57-5d37bd583bf0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:09.612152 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 45/89] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:09 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:09.612991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3120d-6c1d-4869-9f57-5d37bd583bf0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:09.670950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b42fb4-3c65-48e1-a976-a080d9b4d684 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:09.707728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5116a-0e6b-4de0-9178-3d2211836301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:09.708762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5116a-0e6b-4de0-9178-3d2211836301 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:09.709249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5116a-0e6b-4de0-9178-3d2211836301 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:09.709701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5116a-0e6b-4de0-9178-3d2211836301 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:09.711052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5116a-0e6b-4de0-9178-3d2211836301 None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:09.712750 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5116a-0e6b-4de0-9178-3d2211836301 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:09.714691 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 45/90] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:09 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:09.722686 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5116a-0e6b-4de0-9178-3d2211836301 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:09.728375 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3120d-6c1d-4869-9f57-5d37bd583bf0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:09.744226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:09.746356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:09.770824 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a96471-97bc-4759-9605-877f19bf8723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:09.771469 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a96471-97bc-4759-9605-877f19bf8723 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:09.771761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a96471-97bc-4759-9605-877f19bf8723 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:09.772041 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a96471-97bc-4759-9605-877f19bf8723 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:09.838925 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5116a-0e6b-4de0-9178-3d2211836301 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:09.856836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a96471-97bc-4759-9605-877f19bf8723 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:09.858537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a96471-97bc-4759-9605-877f19bf8723 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:09.885897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fd4a9-e175-4a8f-92c1-8751376806ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:09.886539 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fd4a9-e175-4a8f-92c1-8751376806ed None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:09.886827 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fd4a9-e175-4a8f-92c1-8751376806ed None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:09.887108 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fd4a9-e175-4a8f-92c1-8751376806ed None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:09.888282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fd4a9-e175-4a8f-92c1-8751376806ed None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:09.889766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fd4a9-e175-4a8f-92c1-8751376806ed None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:09.891438 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 46/91] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:09 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:09.892279 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fd4a9-e175-4a8f-92c1-8751376806ed None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:09.983111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 46/92] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:09.984377 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a96471-97bc-4759-9605-877f19bf8723 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:10.004247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:10.013049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:10.026264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fd4a9-e175-4a8f-92c1-8751376806ed None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:10.077244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c02959-0c78-40b2-9d83-5513be515c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:10.077909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c02959-0c78-40b2-9d83-5513be515c04 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:10.078428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c02959-0c78-40b2-9d83-5513be515c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:10.078715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c02959-0c78-40b2-9d83-5513be515c04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:10.079455 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c02959-0c78-40b2-9d83-5513be515c04 None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=7849898da2494c4897ab5e0e09c9fe9e, user.id=318df21351594db286347c68d09ec473) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:10.079794 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c02959-0c78-40b2-9d83-5513be515c04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:10.082126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c02959-0c78-40b2-9d83-5513be515c04 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:10.090585 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 47/93] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:09 2022] GET /identity/v3/role_assignments?user.id=318df21351594db286347c68d09ec473&scope.project.id=7849898da2494c4897ab5e0e09c9fe9e&role.id=23538dc4c770454e80dd11ed8e498539 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:10.092280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c02959-0c78-40b2-9d83-5513be515c04 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:10.121531 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a96471-97bc-4759-9605-877f19bf8723 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:10.180613 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c9bfb-6a6b-42af-bcdf-e15bd2257e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:10.181617 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c9bfb-6a6b-42af-bcdf-e15bd2257e38 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:10.182674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c9bfb-6a6b-42af-bcdf-e15bd2257e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:10.183163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c9bfb-6a6b-42af-bcdf-e15bd2257e38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:10.184229 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c9bfb-6a6b-42af-bcdf-e15bd2257e38 None admin] RBAC: Adding query filter params (role.id=ebbe7434d1ec4bc4ba71737f55987dd9, scope.project.id=7849898da2494c4897ab5e0e09c9fe9e, user.id=28ab6d08782b4c26b0afe016bbcd8688) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:10.184894 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c9bfb-6a6b-42af-bcdf-e15bd2257e38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:10.186647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c9bfb-6a6b-42af-bcdf-e15bd2257e38 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:10.204036 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 47/94] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:09 2022] GET /identity/v3/role_assignments?user.id=28ab6d08782b4c26b0afe016bbcd8688&scope.project.id=7849898da2494c4897ab5e0e09c9fe9e&role.id=ebbe7434d1ec4bc4ba71737f55987dd9 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:10.205249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c9bfb-6a6b-42af-bcdf-e15bd2257e38 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:10.303235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:10.338509 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c9bfb-6a6b-42af-bcdf-e15bd2257e38 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:10.355893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c02959-0c78-40b2-9d83-5513be515c04 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:10.408493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b69967-260a-4553-91cc-c0ab064edf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:10.409157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b69967-260a-4553-91cc-c0ab064edf4e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:10.409444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b69967-260a-4553-91cc-c0ab064edf4e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:10.409723 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b69967-260a-4553-91cc-c0ab064edf4e None admin] PATH_INFO: `/v3/projects/dae378bdfac34f76b6cc36e4b2fbae94` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:10.412348 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b69967-260a-4553-91cc-c0ab064edf4e None admin] RBAC: Authorizing `identity:get_project(project_id=dae378bdfac34f76b6cc36e4b2fbae94)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:10.413829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b69967-260a-4553-91cc-c0ab064edf4e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:10.415719 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 48/95] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:10 2022] GET /identity/v3/projects/dae378bdfac34f76b6cc36e4b2fbae94 => generated 312 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:10.422998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b69967-260a-4553-91cc-c0ab064edf4e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:10.426292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a06f4-2261-473e-a9ff-2ab1d464fc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:10.426915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a06f4-2261-473e-a9ff-2ab1d464fc2c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:10.427200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a06f4-2261-473e-a9ff-2ab1d464fc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:10.427478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a06f4-2261-473e-a9ff-2ab1d464fc2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:10.428241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a06f4-2261-473e-a9ff-2ab1d464fc2c None admin] RBAC: Adding query filter params (role.id=31058b9633b44d7ca64f103119987fd9, scope.project.id=7849898da2494c4897ab5e0e09c9fe9e, user.id=28ab6d08782b4c26b0afe016bbcd8688) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:10.428553 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a06f4-2261-473e-a9ff-2ab1d464fc2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:10.430019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a06f4-2261-473e-a9ff-2ab1d464fc2c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:10.444133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 48/96] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:10 2022] GET /identity/v3/role_assignments?user.id=28ab6d08782b4c26b0afe016bbcd8688&scope.project.id=7849898da2494c4897ab5e0e09c9fe9e&role.id=31058b9633b44d7ca64f103119987fd9 => generated 259 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:10.670988 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b69967-260a-4553-91cc-c0ab064edf4e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:10.734076 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85065d-b3a7-4854-b353-0cdc1179b0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:10.734703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85065d-b3a7-4854-b353-0cdc1179b0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:10.734986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85065d-b3a7-4854-b353-0cdc1179b0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:10.735273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85065d-b3a7-4854-b353-0cdc1179b0a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:10.736034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85065d-b3a7-4854-b353-0cdc1179b0a6 None admin] RBAC: Adding query filter params (role.id=ebbe7434d1ec4bc4ba71737f55987dd9, scope.project.id=dae378bdfac34f76b6cc36e4b2fbae94, user.id=28ab6d08782b4c26b0afe016bbcd8688) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:10.736346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85065d-b3a7-4854-b353-0cdc1179b0a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:10.737806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85065d-b3a7-4854-b353-0cdc1179b0a6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:10.743812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 49/97] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:10 2022] GET /identity/v3/role_assignments?user.id=28ab6d08782b4c26b0afe016bbcd8688&scope.project.id=dae378bdfac34f76b6cc36e4b2fbae94&role.id=ebbe7434d1ec4bc4ba71737f55987dd9 => generated 259 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:12.827683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e111b2e-a877-4a6f-b09a-6f1e4b796f73 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:12.828200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e111b2e-a877-4a6f-b09a-6f1e4b796f73 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:12.828307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e111b2e-a877-4a6f-b09a-6f1e4b796f73 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:12.829631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 49/98] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:12.843095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e8829-c15a-4e45-b356-7b6c524995b3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:12.843602 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e8829-c15a-4e45-b356-7b6c524995b3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:12.843679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e8829-c15a-4e45-b356-7b6c524995b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:12.866579 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e74e8829-c15a-4e45-b356-7b6c524995b3 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:12.867879 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e74e8829-c15a-4e45-b356-7b6c524995b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:12.876768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:12.971783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 50/99] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:13.041641 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8891ba4-dde7-4452-8c2f-1c82fefba921 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:13.041955 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8891ba4-dde7-4452-8c2f-1c82fefba921 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:13.042203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8891ba4-dde7-4452-8c2f-1c82fefba921 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:13.043739 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 50/100] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:13.049149 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d7a15-c95a-486e-a4a4-f1b9498deced None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:13.049448 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d7a15-c95a-486e-a4a4-f1b9498deced None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:13.049728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d7a15-c95a-486e-a4a4-f1b9498deced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:13.075931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9b0d7a15-c95a-486e-a4a4-f1b9498deced None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:13.077137 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9b0d7a15-c95a-486e-a4a4-f1b9498deced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:13.095980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b1ec6-fd45-4060-9558-ca52df43c8fc None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:13.096243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b1ec6-fd45-4060-9558-ca52df43c8fc None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:13.096467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b1ec6-fd45-4060-9558-ca52df43c8fc None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:13.097940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 51/101] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:14:13.103267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc74e03-b58c-4ebd-9c4c-9ad102f95771 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:13.103555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc74e03-b58c-4ebd-9c4c-9ad102f95771 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:13.103969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc74e03-b58c-4ebd-9c4c-9ad102f95771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:13.135451 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-2dc74e03-b58c-4ebd-9c4c-9ad102f95771 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:13.136695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2dc74e03-b58c-4ebd-9c4c-9ad102f95771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:13.181177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:13.241065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 51/102] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:13.276412 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 52/103] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:13.364101 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d7a15-c95a-486e-a4a4-f1b9498deced None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:13.437369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:13.471681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d7a15-c95a-486e-a4a4-f1b9498deced None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:13.507523 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a6f09-8f24-4688-b3f5-40c2da239208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:13.508193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a6f09-8f24-4688-b3f5-40c2da239208 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:13.508478 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a6f09-8f24-4688-b3f5-40c2da239208 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:13.508756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a6f09-8f24-4688-b3f5-40c2da239208 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:13.519740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a6f09-8f24-4688-b3f5-40c2da239208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-604a6f09-8f24-4688-b3f5-40c2da239208 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:13.524662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:13.525522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:13.526380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:13.526380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:13.526380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:13.526380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:13.526380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:13.526380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:13.526380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:13.526380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:13.526380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:13.526380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:13.526380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:13.526380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:13.526380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 52/104] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:13.536697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc74e03-b58c-4ebd-9c4c-9ad102f95771 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:13.597431 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59c17e-ee6a-47ed-8753-486ae58fe61f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:13.597721 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59c17e-ee6a-47ed-8753-486ae58fe61f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:13.598005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59c17e-ee6a-47ed-8753-486ae58fe61f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:13.599633 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 53/105] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:13.604907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5800752-7fa9-40eb-9a2d-37dc57613237 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:13.605196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5800752-7fa9-40eb-9a2d-37dc57613237 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:13.605481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5800752-7fa9-40eb-9a2d-37dc57613237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:13.632690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f5800752-7fa9-40eb-9a2d-37dc57613237 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:13.633840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f5800752-7fa9-40eb-9a2d-37dc57613237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:13.677885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc74e03-b58c-4ebd-9c4c-9ad102f95771 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:13.737750 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50e067-edff-4e2d-b400-1ac4a2a78383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:13.738372 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50e067-edff-4e2d-b400-1ac4a2a78383 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:13.738655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50e067-edff-4e2d-b400-1ac4a2a78383 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:13.738927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50e067-edff-4e2d-b400-1ac4a2a78383 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:13.739614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50e067-edff-4e2d-b400-1ac4a2a78383 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:13.739927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50e067-edff-4e2d-b400-1ac4a2a78383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:13.745976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 54/106] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:13.752439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50e067-edff-4e2d-b400-1ac4a2a78383 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:13.754580 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5800752-7fa9-40eb-9a2d-37dc57613237 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:13.773718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 53/107] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:13.774646 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50e067-edff-4e2d-b400-1ac4a2a78383 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:13.805660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:13.836214 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:13.920349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5800752-7fa9-40eb-9a2d-37dc57613237 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:13.958745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50e067-edff-4e2d-b400-1ac4a2a78383 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:13.994972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5d07f-96e9-40f8-b8df-37164fec0bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:13.995161 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b4d4e-2872-4495-8666-bb4cb5157c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:13.995885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b4d4e-2872-4495-8666-bb4cb5157c26 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:13.996120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5d07f-96e9-40f8-b8df-37164fec0bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:13.996411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5d07f-96e9-40f8-b8df-37164fec0bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:13.996692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5d07f-96e9-40f8-b8df-37164fec0bf4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:13.996986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b4d4e-2872-4495-8666-bb4cb5157c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:13.997444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b4d4e-2872-4495-8666-bb4cb5157c26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:14.000267 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5d07f-96e9-40f8-b8df-37164fec0bf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:14.002183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5d07f-96e9-40f8-b8df-37164fec0bf4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:14.006953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 55/108] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:14:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:14.009315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5d07f-96e9-40f8-b8df-37164fec0bf4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:14.023925 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b4d4e-2872-4495-8666-bb4cb5157c26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-e47b4d4e-2872-4495-8666-bb4cb5157c26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:14.033239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:14.034110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:14.035017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:14.035017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:14.035017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:14.035017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:14.035017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:14.035017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:14.035017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:14.035017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:14.035017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:14.035017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:14.035017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:14.035017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:14.043631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 54/109] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:14:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:14.044688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b4d4e-2872-4495-8666-bb4cb5157c26 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:14.123210 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:14.140336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5d07f-96e9-40f8-b8df-37164fec0bf4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:14.156944 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b4d4e-2872-4495-8666-bb4cb5157c26 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:14.170274 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:14.180430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13450f-354a-4586-a107-f53cc0a54ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:14.180988 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13450f-354a-4586-a107-f53cc0a54ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:14.181267 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13450f-354a-4586-a107-f53cc0a54ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:14.181544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13450f-354a-4586-a107-f53cc0a54ee3 None admin] PATH_INFO: `/v3/users/c2f0f8565e6445ce867bb3ea06d9b867` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:14.192051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2d293-e8e3-4005-832b-9cfae234d526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:14.192661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2d293-e8e3-4005-832b-9cfae234d526 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:14.192950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2d293-e8e3-4005-832b-9cfae234d526 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:14.193224 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2d293-e8e3-4005-832b-9cfae234d526 None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:14.193873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f2d293-e8e3-4005-832b-9cfae234d526 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:14.195351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f2d293-e8e3-4005-832b-9cfae234d526 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:14.195770 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13450f-354a-4586-a107-f53cc0a54ee3 None admin] RBAC: Authorizing `identity:get_user(user_id=c2f0f8565e6445ce867bb3ea06d9b867)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:14.197383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13450f-354a-4586-a107-f53cc0a54ee3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:14.199751 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 56/110] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:14 2022] GET /identity/v3/users/c2f0f8565e6445ce867bb3ea06d9b867 => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:14.200694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13450f-354a-4586-a107-f53cc0a54ee3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:14.232654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 55/111] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Sep 12 23:14:14 2022] POST /identity/v3/projects => generated 302 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:14:14.233495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2d293-e8e3-4005-832b-9cfae234d526 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:14.330532 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13450f-354a-4586-a107-f53cc0a54ee3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:14.349160 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2d293-e8e3-4005-832b-9cfae234d526 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:14.371124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbccbb0-3f01-4c91-bd1a-6c64314a687d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:14.371804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbccbb0-3f01-4c91-bd1a-6c64314a687d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:14.372099 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbccbb0-3f01-4c91-bd1a-6c64314a687d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:14.372375 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbccbb0-3f01-4c91-bd1a-6c64314a687d None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:14.373614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbccbb0-3f01-4c91-bd1a-6c64314a687d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:14.374086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbccbb0-3f01-4c91-bd1a-6c64314a687d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:14.375657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbccbb0-3f01-4c91-bd1a-6c64314a687d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:14.376816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:14.390772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 57/112] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:14.390772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5181ec-1c1d-48cc-93a7-cc75853621f7 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:14.390772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5181ec-1c1d-48cc-93a7-cc75853621f7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:14.390772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5181ec-1c1d-48cc-93a7-cc75853621f7 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:14.390772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 58/113] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:14.390772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5181ec-1c1d-48cc-93a7-cc75853621f7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:14.398273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e97a9f-5f06-48fc-a916-ea9f9924d209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:14.399302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e97a9f-5f06-48fc-a916-ea9f9924d209 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:14.399302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e97a9f-5f06-48fc-a916-ea9f9924d209 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:14.399491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e97a9f-5f06-48fc-a916-ea9f9924d209 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:14.413077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e97a9f-5f06-48fc-a916-ea9f9924d209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-e0e97a9f-5f06-48fc-a916-ea9f9924d209 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:14.417630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:14.418506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:14.419265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:14.419265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:14.419265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:14.419265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:14.419265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:14.419265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:14.419265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:14.419265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:14.419265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:14.419265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:14.419265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:14.419265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:14.419265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 56/114] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:14.421627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5dae3-2646-433e-938b-6495693d3e58 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:14.421917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5dae3-2646-433e-938b-6495693d3e58 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:14.422193 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5dae3-2646-433e-938b-6495693d3e58 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:14.424424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 57/115] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:14.425206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5dae3-2646-433e-938b-6495693d3e58 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:14.451899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:14.508730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5181ec-1c1d-48cc-93a7-cc75853621f7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:14.540177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5dae3-2646-433e-938b-6495693d3e58 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:14.544786 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ce569-4261-4c38-aec6-9d3c3225d44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:14.545451 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ce569-4261-4c38-aec6-9d3c3225d44a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:14.545737 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ce569-4261-4c38-aec6-9d3c3225d44a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:14.546015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ce569-4261-4c38-aec6-9d3c3225d44a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:14.576628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a66be0-6ebb-44b5-9b9d-19d0a111c8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:14.577261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a66be0-6ebb-44b5-9b9d-19d0a111c8df None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:14.577566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a66be0-6ebb-44b5-9b9d-19d0a111c8df None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:14.577851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a66be0-6ebb-44b5-9b9d-19d0a111c8df None admin] PATH_INFO: `/v3/users/318df21351594db286347c68d09ec473` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:14.578933 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a66be0-6ebb-44b5-9b9d-19d0a111c8df None admin] RBAC: Authorizing `identity:get_user(user_id=318df21351594db286347c68d09ec473)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:14.580517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a66be0-6ebb-44b5-9b9d-19d0a111c8df None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:14.582282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 58/116] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:14 2022] GET /identity/v3/users/318df21351594db286347c68d09ec473 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:14.587663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d779b4-b822-414e-a6e1-9684be9f43d3 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:14.588003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d779b4-b822-414e-a6e1-9684be9f43d3 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:14.588365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d779b4-b822-414e-a6e1-9684be9f43d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:14.606425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-a2d779b4-b822-414e-a6e1-9684be9f43d3 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:14.607644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a2d779b4-b822-414e-a6e1-9684be9f43d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:14.618625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ce569-4261-4c38-aec6-9d3c3225d44a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:14.620645 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ce569-4261-4c38-aec6-9d3c3225d44a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:14.657502 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:14.672484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:14.678677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 59/117] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:14.679762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d779b4-b822-414e-a6e1-9684be9f43d3 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:14.757491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 59/118] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:14.757491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c439f-9d32-42b5-a4bc-d32a7a26d738 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:14.757491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c439f-9d32-42b5-a4bc-d32a7a26d738 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:14.757491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c439f-9d32-42b5-a4bc-d32a7a26d738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:14.777051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2e4c439f-9d32-42b5-a4bc-d32a7a26d738 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:14.779117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2e4c439f-9d32-42b5-a4bc-d32a7a26d738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:14.797741 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d779b4-b822-414e-a6e1-9684be9f43d3 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:14.833591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3582422-4bd2-4d69-95a9-4992ced666a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:14.834426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3582422-4bd2-4d69-95a9-4992ced666a4 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:14.834852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3582422-4bd2-4d69-95a9-4992ced666a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:14.835267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3582422-4bd2-4d69-95a9-4992ced666a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:14.836128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3582422-4bd2-4d69-95a9-4992ced666a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:14.836576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3582422-4bd2-4d69-95a9-4992ced666a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:14.838159 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3582422-4bd2-4d69-95a9-4992ced666a4 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:14.846708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 60/119] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:14.847914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3582422-4bd2-4d69-95a9-4992ced666a4 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:14.868757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 60/120] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:14.870033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c439f-9d32-42b5-a4bc-d32a7a26d738 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:14.905412 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:14.960747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:14.986637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c439f-9d32-42b5-a4bc-d32a7a26d738 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:15.025321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4f579-6d2d-45a1-8a40-9021edbb362a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:15.026232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4f579-6d2d-45a1-8a40-9021edbb362a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:15.026665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4f579-6d2d-45a1-8a40-9021edbb362a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:15.027312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4f579-6d2d-45a1-8a40-9021edbb362a None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:15.029292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4f579-6d2d-45a1-8a40-9021edbb362a None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:15.030624 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3582422-4bd2-4d69-95a9-4992ced666a4 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:15.031387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4f579-6d2d-45a1-8a40-9021edbb362a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:15.033974 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 61/121] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:14 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:15.034892 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4f579-6d2d-45a1-8a40-9021edbb362a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:15.065698 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87a19e-a5ee-418c-81f4-0aa542e64d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:15.066339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87a19e-a5ee-418c-81f4-0aa542e64d61 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:15.066624 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87a19e-a5ee-418c-81f4-0aa542e64d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:15.066894 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87a19e-a5ee-418c-81f4-0aa542e64d61 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:15.068059 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87a19e-a5ee-418c-81f4-0aa542e64d61 None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:15.069492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87a19e-a5ee-418c-81f4-0aa542e64d61 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:15.071181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 61/122] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:14 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:15.077466 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87a19e-a5ee-418c-81f4-0aa542e64d61 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:15.154095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4f579-6d2d-45a1-8a40-9021edbb362a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:15.189869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87a19e-a5ee-418c-81f4-0aa542e64d61 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:15.194327 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e74a9-6737-453c-84d4-e4c22a5d0b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:15.194963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e74a9-6737-453c-84d4-e4c22a5d0b33 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:15.195247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e74a9-6737-453c-84d4-e4c22a5d0b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:15.195524 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e74a9-6737-453c-84d4-e4c22a5d0b33 None admin] PATH_INFO: `/v3/users/28ab6d08782b4c26b0afe016bbcd8688` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:15.196708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e74a9-6737-453c-84d4-e4c22a5d0b33 None admin] RBAC: Authorizing `identity:get_user(user_id=28ab6d08782b4c26b0afe016bbcd8688)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:15.198247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e74a9-6737-453c-84d4-e4c22a5d0b33 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:15.199869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 62/123] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:15 2022] GET /identity/v3/users/28ab6d08782b4c26b0afe016bbcd8688 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:15.200694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e74a9-6737-453c-84d4-e4c22a5d0b33 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:15.205820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:15.216628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:15.234650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8310e3-8563-46d6-8ca9-d821258ca1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:15.235281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8310e3-8563-46d6-8ca9-d821258ca1e4 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:15.235563 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8310e3-8563-46d6-8ca9-d821258ca1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:15.235866 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8310e3-8563-46d6-8ca9-d821258ca1e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:15.246156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8310e3-8563-46d6-8ca9-d821258ca1e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-3c8310e3-8563-46d6-8ca9-d821258ca1e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:15.251425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:15.252496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:15.253249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:15.253249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:15.253249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:15.253249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:15.253249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:15.253249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:15.253249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:15.253249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:15.253249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:15.253249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:15.253249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:15.253249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:15.253249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 62/124] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:15.253875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8310e3-8563-46d6-8ca9-d821258ca1e4 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:15.307885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e74a9-6737-453c-84d4-e4c22a5d0b33 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:15.345512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d7fb4-408d-47ac-98ad-6970bd07e53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:15.346109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d7fb4-408d-47ac-98ad-6970bd07e53d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:15.346391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d7fb4-408d-47ac-98ad-6970bd07e53d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:15.346671 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d7fb4-408d-47ac-98ad-6970bd07e53d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:15.347392 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d7fb4-408d-47ac-98ad-6970bd07e53d None admin] RBAC: Adding query filter params (role.id=c6bc2b1713e84fecb6708753f6ba6a66, scope.project.id=7849898da2494c4897ab5e0e09c9fe9e, user.id=c2f0f8565e6445ce867bb3ea06d9b867) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:15.347731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d7fb4-408d-47ac-98ad-6970bd07e53d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:15.349153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d7fb4-408d-47ac-98ad-6970bd07e53d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:15.354422 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 63/125] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:15 2022] GET /identity/v3/role_assignments?user.id=c2f0f8565e6445ce867bb3ea06d9b867&scope.project.id=7849898da2494c4897ab5e0e09c9fe9e&role.id=c6bc2b1713e84fecb6708753f6ba6a66 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:15.355194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d7fb4-408d-47ac-98ad-6970bd07e53d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:15.366646 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8310e3-8563-46d6-8ca9-d821258ca1e4 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:15.409766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968f8fa-06ef-4250-bf6e-a4e3c91a0f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:15.411173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968f8fa-06ef-4250-bf6e-a4e3c91a0f6f None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:15.412023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968f8fa-06ef-4250-bf6e-a4e3c91a0f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:15.412446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968f8fa-06ef-4250-bf6e-a4e3c91a0f6f None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e/users/318df21351594db286347c68d09ec473/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:15.415168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968f8fa-06ef-4250-bf6e-a4e3c91a0f6f None admin] RBAC: Authorizing `identity:create_grant(project_id=7849898da2494c4897ab5e0e09c9fe9e, user_id=318df21351594db286347c68d09ec473, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:15.418312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968f8fa-06ef-4250-bf6e-a4e3c91a0f6f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:15.437488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 63/126] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:14:15 2022] PUT /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e/users/318df21351594db286347c68d09ec473/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:14:15.444719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968f8fa-06ef-4250-bf6e-a4e3c91a0f6f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:15.462638 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:15.492807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d7fb4-408d-47ac-98ad-6970bd07e53d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:15.516427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:15.536639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026535aa-b8af-4c21-8da5-3bcc2357c093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:15.537696 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026535aa-b8af-4c21-8da5-3bcc2357c093 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:15.538179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026535aa-b8af-4c21-8da5-3bcc2357c093 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:15.538824 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026535aa-b8af-4c21-8da5-3bcc2357c093 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:15.552980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026535aa-b8af-4c21-8da5-3bcc2357c093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-026535aa-b8af-4c21-8da5-3bcc2357c093 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:15.561535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:15.562487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:15.563259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:15.563259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:15.563259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:15.563259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:15.563259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:15.563259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:15.563259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:15.563259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:15.563259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:15.563259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:15.563259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:15.563259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:15.564925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 64/127] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:14:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:15.569159 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026535aa-b8af-4c21-8da5-3bcc2357c093 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:15.637167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968f8fa-06ef-4250-bf6e-a4e3c91a0f6f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:15.700524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a08cc-9515-4e45-9dea-0410c1dcef72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:15.702510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a08cc-9515-4e45-9dea-0410c1dcef72 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:15.702510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a08cc-9515-4e45-9dea-0410c1dcef72 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:15.702510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a08cc-9515-4e45-9dea-0410c1dcef72 None admin] PATH_INFO: `/v3/projects/dae378bdfac34f76b6cc36e4b2fbae94` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:15.705468 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a08cc-9515-4e45-9dea-0410c1dcef72 None admin] RBAC: Authorizing `identity:get_project(project_id=dae378bdfac34f76b6cc36e4b2fbae94)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:15.710457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a08cc-9515-4e45-9dea-0410c1dcef72 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:15.714811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 64/128] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:15 2022] GET /identity/v3/projects/dae378bdfac34f76b6cc36e4b2fbae94 => generated 312 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:15.715667 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a08cc-9515-4e45-9dea-0410c1dcef72 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:15.718138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026535aa-b8af-4c21-8da5-3bcc2357c093 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:15.754505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b237a-3af3-45ab-be33-12db3f5e2a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:15.755123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b237a-3af3-45ab-be33-12db3f5e2a19 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:15.755409 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b237a-3af3-45ab-be33-12db3f5e2a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:15.755716 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b237a-3af3-45ab-be33-12db3f5e2a19 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:15.756388 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b237a-3af3-45ab-be33-12db3f5e2a19 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:15.756722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b237a-3af3-45ab-be33-12db3f5e2a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:15.758087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b237a-3af3-45ab-be33-12db3f5e2a19 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:15.765226 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:15.769873 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 65/129] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:15.770897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b237a-3af3-45ab-be33-12db3f5e2a19 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:15.847623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:15.852776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a08cc-9515-4e45-9dea-0410c1dcef72 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:15.882144 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b237a-3af3-45ab-be33-12db3f5e2a19 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:15.898182 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692af0-8abe-4c7b-9c50-95e1d869dc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:15.899583 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692af0-8abe-4c7b-9c50-95e1d869dc21 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:15.900471 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692af0-8abe-4c7b-9c50-95e1d869dc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:15.901009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692af0-8abe-4c7b-9c50-95e1d869dc21 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:15.902116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd692af0-8abe-4c7b-9c50-95e1d869dc21 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:15.902688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd692af0-8abe-4c7b-9c50-95e1d869dc21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:15.905238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd692af0-8abe-4c7b-9c50-95e1d869dc21 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:15.917243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 65/130] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:14:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:15.919079 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692af0-8abe-4c7b-9c50-95e1d869dc21 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:15.955706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ed1d6-5d41-4cd8-a9b4-11dae6745308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:15.955706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ed1d6-5d41-4cd8-a9b4-11dae6745308 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:15.955706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ed1d6-5d41-4cd8-a9b4-11dae6745308 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:15.955706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ed1d6-5d41-4cd8-a9b4-11dae6745308 None admin] PATH_INFO: `/v3/projects/dae378bdfac34f76b6cc36e4b2fbae94/users/28ab6d08782b4c26b0afe016bbcd8688/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:15.957309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ed1d6-5d41-4cd8-a9b4-11dae6745308 None admin] RBAC: Authorizing `identity:create_grant(project_id=dae378bdfac34f76b6cc36e4b2fbae94, user_id=28ab6d08782b4c26b0afe016bbcd8688, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:15.959609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ed1d6-5d41-4cd8-a9b4-11dae6745308 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:15.989352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 66/131] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:14:15 2022] PUT /identity/v3/projects/dae378bdfac34f76b6cc36e4b2fbae94/users/28ab6d08782b4c26b0afe016bbcd8688/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:14:15.991215 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ed1d6-5d41-4cd8-a9b4-11dae6745308 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:16.138613 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:16.171231 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692af0-8abe-4c7b-9c50-95e1d869dc21 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:16.216558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ed1d6-5d41-4cd8-a9b4-11dae6745308 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:16.234079 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c612be-406c-46a6-a209-4359b29d9f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:16.234737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c612be-406c-46a6-a209-4359b29d9f39 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:16.235020 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c612be-406c-46a6-a209-4359b29d9f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:16.235296 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c612be-406c-46a6-a209-4359b29d9f39 None admin] PATH_INFO: `/v3/users/28ab6d08782b4c26b0afe016bbcd8688` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:16.236692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c612be-406c-46a6-a209-4359b29d9f39 None admin] RBAC: Authorizing `identity:get_user(user_id=28ab6d08782b4c26b0afe016bbcd8688)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:16.238140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c612be-406c-46a6-a209-4359b29d9f39 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:16.242777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 66/132] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:15 2022] GET /identity/v3/users/28ab6d08782b4c26b0afe016bbcd8688 => generated 275 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:16.246307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c612be-406c-46a6-a209-4359b29d9f39 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:16.285555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119be47-7d37-4ac8-8f22-aec0d77aed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:16.286197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119be47-7d37-4ac8-8f22-aec0d77aed31 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:16.286484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119be47-7d37-4ac8-8f22-aec0d77aed31 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:16.286760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119be47-7d37-4ac8-8f22-aec0d77aed31 None admin] PATH_INFO: `/v3/users/28ab6d08782b4c26b0afe016bbcd8688` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:16.288789 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119be47-7d37-4ac8-8f22-aec0d77aed31 None admin] RBAC: Authorizing `identity:get_user(user_id=28ab6d08782b4c26b0afe016bbcd8688)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:16.290241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119be47-7d37-4ac8-8f22-aec0d77aed31 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:16.292352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 67/133] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:15 2022] GET /identity/v3/users/28ab6d08782b4c26b0afe016bbcd8688 => generated 275 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:16.303533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:16.304468 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119be47-7d37-4ac8-8f22-aec0d77aed31 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:16.396353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c612be-406c-46a6-a209-4359b29d9f39 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:16.437133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59760ac-629e-42a7-959f-01f76c3f6789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:16.437782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59760ac-629e-42a7-959f-01f76c3f6789 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:16.438070 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59760ac-629e-42a7-959f-01f76c3f6789 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:16.438351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59760ac-629e-42a7-959f-01f76c3f6789 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:16.439510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59760ac-629e-42a7-959f-01f76c3f6789 None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:16.441523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59760ac-629e-42a7-959f-01f76c3f6789 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:16.443870 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 67/134] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:16 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:16.450164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59760ac-629e-42a7-959f-01f76c3f6789 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:16.474486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119be47-7d37-4ac8-8f22-aec0d77aed31 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:16.515835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:16.525227 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a577dc-31bb-43a3-ab95-d85266da6bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:16.525953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a577dc-31bb-43a3-ab95-d85266da6bbf None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:16.526299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a577dc-31bb-43a3-ab95-d85266da6bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:16.526588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a577dc-31bb-43a3-ab95-d85266da6bbf None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:16.527820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a577dc-31bb-43a3-ab95-d85266da6bbf None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:16.529315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a577dc-31bb-43a3-ab95-d85266da6bbf None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:16.531194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 68/135] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:16 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:16.537830 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a577dc-31bb-43a3-ab95-d85266da6bbf None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:16.653171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a577dc-31bb-43a3-ab95-d85266da6bbf None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:16.690405 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3e6ea-deef-42bc-a724-802893609ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:16.691089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3e6ea-deef-42bc-a724-802893609ba4 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:16.691378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3e6ea-deef-42bc-a724-802893609ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:16.691685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3e6ea-deef-42bc-a724-802893609ba4 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e/users/28ab6d08782b4c26b0afe016bbcd8688/roles/31058b9633b44d7ca64f103119987fd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:16.693909 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3e6ea-deef-42bc-a724-802893609ba4 None admin] RBAC: Authorizing `identity:create_grant(project_id=7849898da2494c4897ab5e0e09c9fe9e, user_id=28ab6d08782b4c26b0afe016bbcd8688, role_id=31058b9633b44d7ca64f103119987fd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:16.695361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3e6ea-deef-42bc-a724-802893609ba4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:16.697997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:16.705493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59760ac-629e-42a7-959f-01f76c3f6789 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:16.712715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 69/136] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:14:16 2022] PUT /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e/users/28ab6d08782b4c26b0afe016bbcd8688/roles/31058b9633b44d7ca64f103119987fd9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:14:16.714571 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c09dc-5a7a-4861-8f2f-6b1ef7cfac8c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:16.714952 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c09dc-5a7a-4861-8f2f-6b1ef7cfac8c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:16.715240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c09dc-5a7a-4861-8f2f-6b1ef7cfac8c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:16.716747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 70/137] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:16.722546 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209de932-2ba4-4ad2-a73a-9787de14bfb8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:16.722844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209de932-2ba4-4ad2-a73a-9787de14bfb8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:16.723132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209de932-2ba4-4ad2-a73a-9787de14bfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:16.749185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-209de932-2ba4-4ad2-a73a-9787de14bfb8 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:16.750481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-209de932-2ba4-4ad2-a73a-9787de14bfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:16.821363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4b568-9bdd-4471-a870-42a9b003c879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:16.822061 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b568-9bdd-4471-a870-42a9b003c879 None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:16.822348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b568-9bdd-4471-a870-42a9b003c879 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:16.822706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b568-9bdd-4471-a870-42a9b003c879 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e/users/28ab6d08782b4c26b0afe016bbcd8688/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:16.824760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4b568-9bdd-4471-a870-42a9b003c879 None admin] RBAC: Authorizing `identity:create_grant(project_id=7849898da2494c4897ab5e0e09c9fe9e, user_id=28ab6d08782b4c26b0afe016bbcd8688, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:16.826244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4b568-9bdd-4471-a870-42a9b003c879 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:16.835262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:16.863940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 71/138] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:16.870369 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 68/139] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:14:16 2022] PUT /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e/users/28ab6d08782b4c26b0afe016bbcd8688/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:14:17.291726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209de932-2ba4-4ad2-a73a-9787de14bfb8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:17.421194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209de932-2ba4-4ad2-a73a-9787de14bfb8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:17.461275 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89aa83-b7cb-448c-af46-d3be106d1ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:17.461909 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89aa83-b7cb-448c-af46-d3be106d1ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:17.462197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89aa83-b7cb-448c-af46-d3be106d1ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:17.462468 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89aa83-b7cb-448c-af46-d3be106d1ef4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:17.466408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89aa83-b7cb-448c-af46-d3be106d1ef4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:17.467891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89aa83-b7cb-448c-af46-d3be106d1ef4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:17.469480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 72/140] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:14:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:17.475112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4b568-9bdd-4471-a870-42a9b003c879 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:17.648826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4b568-9bdd-4471-a870-42a9b003c879 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:17.694786 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824965bc-e62a-4588-a73d-658332d86f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:17.695799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824965bc-e62a-4588-a73d-658332d86f21 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:17.696296 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824965bc-e62a-4588-a73d-658332d86f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:17.696776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824965bc-e62a-4588-a73d-658332d86f21 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:17.697704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824965bc-e62a-4588-a73d-658332d86f21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:17.699448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824965bc-e62a-4588-a73d-658332d86f21 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:17.726849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 69/141] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:14:17 2022] POST /identity/v3/users => generated 283 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:14:18.362785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecc438-5175-4711-b38c-4a223d63eadb None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:18.363273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecc438-5175-4711-b38c-4a223d63eadb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:18.363273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecc438-5175-4711-b38c-4a223d63eadb None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:18.364640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 73/142] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:18.370506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e00ed-80c1-41ec-9486-0932f7d07a99 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:18.370805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e00ed-80c1-41ec-9486-0932f7d07a99 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:18.371089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e00ed-80c1-41ec-9486-0932f7d07a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:18.401131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-292e00ed-80c1-41ec-9486-0932f7d07a99 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:18.402376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-292e00ed-80c1-41ec-9486-0932f7d07a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:18.413378 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:18.490290 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 70/143] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:18.604399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7bfc1-b848-42f9-b4f0-6be1bd0a0f94 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:18.604755 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7bfc1-b848-42f9-b4f0-6be1bd0a0f94 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:18.604966 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7bfc1-b848-42f9-b4f0-6be1bd0a0f94 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:18.606473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 74/144] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:18.614864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695b30f-c258-4b78-ab26-094df111b000 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:18.615091 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695b30f-c258-4b78-ab26-094df111b000 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:18.615375 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695b30f-c258-4b78-ab26-094df111b000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:18.644579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-8695b30f-c258-4b78-ab26-094df111b000 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:18.645779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8695b30f-c258-4b78-ab26-094df111b000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:18.721818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 71/145] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:18.914972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7bfc1-b848-42f9-b4f0-6be1bd0a0f94 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:18.985278 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:19.036880 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7bfc1-b848-42f9-b4f0-6be1bd0a0f94 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:19.067086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695b30f-c258-4b78-ab26-094df111b000 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:19.079077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e53bb9-3220-475e-8613-b2b1beb48dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:19.079761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e53bb9-3220-475e-8613-b2b1beb48dbc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:19.080047 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e53bb9-3220-475e-8613-b2b1beb48dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:19.080482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e53bb9-3220-475e-8613-b2b1beb48dbc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:19.096311 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e53bb9-3220-475e-8613-b2b1beb48dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e2e53bb9-3220-475e-8613-b2b1beb48dbc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:19.101305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:19.102193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:19.102989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:19.102989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:19.102989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:19.102989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:19.102989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:19.102989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:19.102989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:19.102989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:19.102989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:19.102989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:19.102989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:19.102989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:19.102989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 75/146] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:19.108291 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e53bb9-3220-475e-8613-b2b1beb48dbc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:19.251761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e53bb9-3220-475e-8613-b2b1beb48dbc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:19.276299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:19.279009 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:19.311751 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c644c5-e019-47e1-980f-f4a1adfafc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:19.312387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c644c5-e019-47e1-980f-f4a1adfafc2d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:19.312673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c644c5-e019-47e1-980f-f4a1adfafc2d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:19.312949 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c644c5-e019-47e1-980f-f4a1adfafc2d None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:19.313630 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c644c5-e019-47e1-980f-f4a1adfafc2d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:19.313937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c644c5-e019-47e1-980f-f4a1adfafc2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:19.315333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c644c5-e019-47e1-980f-f4a1adfafc2d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:19.329539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 76/147] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:19.336629 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695b30f-c258-4b78-ab26-094df111b000 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:19.348882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c644c5-e019-47e1-980f-f4a1adfafc2d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:19.404327 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f33be9-bdc0-4b40-b364-b4a2bd4ed711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:19.405107 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f33be9-bdc0-4b40-b364-b4a2bd4ed711 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:19.405398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f33be9-bdc0-4b40-b364-b4a2bd4ed711 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:19.405678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f33be9-bdc0-4b40-b364-b4a2bd4ed711 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:19.429466 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f33be9-bdc0-4b40-b364-b4a2bd4ed711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-95f33be9-bdc0-4b40-b364-b4a2bd4ed711 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:19.434995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:19.435912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:19.436677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:19.436677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:19.436677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:19.436677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:19.436677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:19.436677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:19.436677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:19.436677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:19.436677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:19.436677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:19.436677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:19.436677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:19.437721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 72/148] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:14:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:19.442017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7da8a-4dc3-4dda-b10d-428b574faaa5 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:19.442311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7da8a-4dc3-4dda-b10d-428b574faaa5 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:19.442630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7da8a-4dc3-4dda-b10d-428b574faaa5 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:19.444796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 73/149] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:19.454198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54322fb-49d7-41ab-a881-86383976297a None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:19.454505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54322fb-49d7-41ab-a881-86383976297a None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:19.454767 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54322fb-49d7-41ab-a881-86383976297a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:19.498123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-e54322fb-49d7-41ab-a881-86383976297a None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:19.499344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e54322fb-49d7-41ab-a881-86383976297a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:19.554068 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c644c5-e019-47e1-980f-f4a1adfafc2d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:19.629685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149d7df-8463-4d26-8a6c-b7425fde3816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:19.630353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149d7df-8463-4d26-8a6c-b7425fde3816 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:19.630640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149d7df-8463-4d26-8a6c-b7425fde3816 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:19.630920 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149d7df-8463-4d26-8a6c-b7425fde3816 None admin] PATH_INFO: `/v3/users/c2f0f8565e6445ce867bb3ea06d9b867` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:19.632226 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149d7df-8463-4d26-8a6c-b7425fde3816 None admin] RBAC: Authorizing `identity:get_user(user_id=c2f0f8565e6445ce867bb3ea06d9b867)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:19.633687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149d7df-8463-4d26-8a6c-b7425fde3816 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:19.634161 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 74/150] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:19.635602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54322fb-49d7-41ab-a881-86383976297a None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:19.636167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 77/151] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:19 2022] GET /identity/v3/users/c2f0f8565e6445ce867bb3ea06d9b867 => generated 289 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:19.644552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149d7df-8463-4d26-8a6c-b7425fde3816 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:19.723644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:19.752317 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:19.808127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54322fb-49d7-41ab-a881-86383976297a None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:19.832780 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149d7df-8463-4d26-8a6c-b7425fde3816 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:19.876311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b72e73-f8ae-43f5-9a80-e7ab4b56eb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:19.876942 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b72e73-f8ae-43f5-9a80-e7ab4b56eb8f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:19.877228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b72e73-f8ae-43f5-9a80-e7ab4b56eb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:19.877509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b72e73-f8ae-43f5-9a80-e7ab4b56eb8f None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:19.878187 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b72e73-f8ae-43f5-9a80-e7ab4b56eb8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:19.878496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b72e73-f8ae-43f5-9a80-e7ab4b56eb8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:19.880008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b72e73-f8ae-43f5-9a80-e7ab4b56eb8f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:19.896060 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 75/152] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:19.897058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b72e73-f8ae-43f5-9a80-e7ab4b56eb8f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:19.908335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26cdab-62c2-4c8f-ab92-fb3dcb271d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:19.909111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26cdab-62c2-4c8f-ab92-fb3dcb271d4d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:19.909398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26cdab-62c2-4c8f-ab92-fb3dcb271d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:19.909674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26cdab-62c2-4c8f-ab92-fb3dcb271d4d None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:19.910865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26cdab-62c2-4c8f-ab92-fb3dcb271d4d None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:19.912467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26cdab-62c2-4c8f-ab92-fb3dcb271d4d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:19.914246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 78/153] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:19 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:19.928216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26cdab-62c2-4c8f-ab92-fb3dcb271d4d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:20.125390 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26cdab-62c2-4c8f-ab92-fb3dcb271d4d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:20.140503 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:20.141547 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b72e73-f8ae-43f5-9a80-e7ab4b56eb8f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:20.178665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ab848-4cc8-472f-91b2-d864d82c41a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:20.179771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ab848-4cc8-472f-91b2-d864d82c41a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:20.180260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ab848-4cc8-472f-91b2-d864d82c41a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:20.180727 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ab848-4cc8-472f-91b2-d864d82c41a2 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e/users/c2f0f8565e6445ce867bb3ea06d9b867/roles/c6bc2b1713e84fecb6708753f6ba6a66` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:20.188255 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ab848-4cc8-472f-91b2-d864d82c41a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=7849898da2494c4897ab5e0e09c9fe9e, user_id=c2f0f8565e6445ce867bb3ea06d9b867, role_id=c6bc2b1713e84fecb6708753f6ba6a66)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:20.189979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ab848-4cc8-472f-91b2-d864d82c41a2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:20.195956 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:20.212071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef9f60-a9da-46ad-9815-14f0914f6adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:20.213338 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef9f60-a9da-46ad-9815-14f0914f6adb None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:20.213875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef9f60-a9da-46ad-9815-14f0914f6adb None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:20.214354 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef9f60-a9da-46ad-9815-14f0914f6adb None admin] PATH_INFO: `/v3/users/318df21351594db286347c68d09ec473` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:20.215615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 79/154] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:14:19 2022] PUT /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e/users/c2f0f8565e6445ce867bb3ea06d9b867/roles/c6bc2b1713e84fecb6708753f6ba6a66 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:14:20.216601 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef9f60-a9da-46ad-9815-14f0914f6adb None admin] RBAC: Authorizing `identity:get_user(user_id=318df21351594db286347c68d09ec473)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:20.218565 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef9f60-a9da-46ad-9815-14f0914f6adb None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:20.220835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 76/155] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:19 2022] GET /identity/v3/users/318df21351594db286347c68d09ec473 => generated 247 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:20.225614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ab848-4cc8-472f-91b2-d864d82c41a2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:20.226967 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef9f60-a9da-46ad-9815-14f0914f6adb None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:20.413378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ab848-4cc8-472f-91b2-d864d82c41a2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:20.445737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef9f60-a9da-46ad-9815-14f0914f6adb None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:20.457183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea7260-7e61-42ee-bccd-977118c79068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:20.457897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea7260-7e61-42ee-bccd-977118c79068 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:20.458183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea7260-7e61-42ee-bccd-977118c79068 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:20.458457 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea7260-7e61-42ee-bccd-977118c79068 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:20.477902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea7260-7e61-42ee-bccd-977118c79068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-42ea7260-7e61-42ee-bccd-977118c79068 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:20.483488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:20.484597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:20.485443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:20.485443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:20.485443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:20.485443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:20.485443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:20.485443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:20.485443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:20.485443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:20.485443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:20.485443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:20.485443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:20.485443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:20.488909 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 80/156] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:20.501964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea7260-7e61-42ee-bccd-977118c79068 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:20.527714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1715f-77ab-4bda-8c3d-92742eeb3f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:20.529122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1715f-77ab-4bda-8c3d-92742eeb3f72 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:20.529606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1715f-77ab-4bda-8c3d-92742eeb3f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:20.530088 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1715f-77ab-4bda-8c3d-92742eeb3f72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:20.550595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:20.633212 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea7260-7e61-42ee-bccd-977118c79068 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:20.678243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353f117-aa02-46d7-9e16-a340764416a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:20.678835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353f117-aa02-46d7-9e16-a340764416a3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:20.679118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353f117-aa02-46d7-9e16-a340764416a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:20.679442 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353f117-aa02-46d7-9e16-a340764416a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:20.680142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353f117-aa02-46d7-9e16-a340764416a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:20.680446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353f117-aa02-46d7-9e16-a340764416a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:20.684266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353f117-aa02-46d7-9e16-a340764416a3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:20.691916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:20.693705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 81/157] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:20.695015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb9669-d293-4d1a-8ca5-1104267dc5b4 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:20.695423 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb9669-d293-4d1a-8ca5-1104267dc5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:20.695729 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb9669-d293-4d1a-8ca5-1104267dc5b4 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:20.696935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 82/158] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:20.698354 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311f6af-fa24-436e-bf53-dba9c5315998 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:20.698733 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311f6af-fa24-436e-bf53-dba9c5315998 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:20.699229 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311f6af-fa24-436e-bf53-dba9c5315998 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:20.700621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 83/159] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:20.704877 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5741552-7ca9-4583-9136-3a323da27b5e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:20.705201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5741552-7ca9-4583-9136-3a323da27b5e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:20.705479 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5741552-7ca9-4583-9136-3a323da27b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:20.707595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1715f-77ab-4bda-8c3d-92742eeb3f72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:20.709106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1715f-77ab-4bda-8c3d-92742eeb3f72 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:20.733480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b5741552-7ca9-4583-9136-3a323da27b5e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:20.734750 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b5741552-7ca9-4583-9136-3a323da27b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:20.833001 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 84/160] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:20.833884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5741552-7ca9-4583-9136-3a323da27b5e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:20.870015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:20.979328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5741552-7ca9-4583-9136-3a323da27b5e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:21.002531 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 77/161] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 776 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:21.011950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c87021-7d50-45ea-ba3d-e809c575bcb6 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:21.012262 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c87021-7d50-45ea-ba3d-e809c575bcb6 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:21.013663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c87021-7d50-45ea-ba3d-e809c575bcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:21.033533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b129d59-d12d-4f06-93d1-86e47eaf7ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:21.034189 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b129d59-d12d-4f06-93d1-86e47eaf7ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:21.034475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b129d59-d12d-4f06-93d1-86e47eaf7ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:21.034752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b129d59-d12d-4f06-93d1-86e47eaf7ac6 None admin] PATH_INFO: `/v3/users/28ab6d08782b4c26b0afe016bbcd8688` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:21.035947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b129d59-d12d-4f06-93d1-86e47eaf7ac6 None admin] RBAC: Authorizing `identity:get_user(user_id=28ab6d08782b4c26b0afe016bbcd8688)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:21.037392 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b129d59-d12d-4f06-93d1-86e47eaf7ac6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:21.039000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 85/162] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:20 2022] GET /identity/v3/users/28ab6d08782b4c26b0afe016bbcd8688 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:21.039795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b129d59-d12d-4f06-93d1-86e47eaf7ac6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:21.060794 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-a7c87021-7d50-45ea-ba3d-e809c575bcb6 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:21.062011 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a7c87021-7d50-45ea-ba3d-e809c575bcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:21.168881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 78/163] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:21.169835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c87021-7d50-45ea-ba3d-e809c575bcb6 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:21.204489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:21.297031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:21.310373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c87021-7d50-45ea-ba3d-e809c575bcb6 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:21.321728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b129d59-d12d-4f06-93d1-86e47eaf7ac6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:21.369066 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3025b51-4f44-4280-bbfb-5405c8c6d392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:21.370294 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3025b51-4f44-4280-bbfb-5405c8c6d392 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:21.370579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3025b51-4f44-4280-bbfb-5405c8c6d392 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:21.370852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3025b51-4f44-4280-bbfb-5405c8c6d392 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:21.383046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ca1d6-4f6b-4c59-b185-4bc3893873e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:21.383738 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ca1d6-4f6b-4c59-b185-4bc3893873e4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:21.384028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ca1d6-4f6b-4c59-b185-4bc3893873e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:21.384479 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ca1d6-4f6b-4c59-b185-4bc3893873e4 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:21.385773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ca1d6-4f6b-4c59-b185-4bc3893873e4 None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:21.387218 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ca1d6-4f6b-4c59-b185-4bc3893873e4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:21.392988 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 86/164] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:21 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:21.398861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ca1d6-4f6b-4c59-b185-4bc3893873e4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:21.466317 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3025b51-4f44-4280-bbfb-5405c8c6d392 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:21.471915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3025b51-4f44-4280-bbfb-5405c8c6d392 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:21.515958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:21.626102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 79/165] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:21.627982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3025b51-4f44-4280-bbfb-5405c8c6d392 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:21.649541 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ca1d6-4f6b-4c59-b185-4bc3893873e4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:21.753448 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eab504-57ad-4d43-b2ba-105a9104dd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:21.754094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eab504-57ad-4d43-b2ba-105a9104dd93 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:21.754375 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eab504-57ad-4d43-b2ba-105a9104dd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:21.754649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eab504-57ad-4d43-b2ba-105a9104dd93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:21.755402 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eab504-57ad-4d43-b2ba-105a9104dd93 None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=7849898da2494c4897ab5e0e09c9fe9e, user.id=318df21351594db286347c68d09ec473) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:21.755751 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eab504-57ad-4d43-b2ba-105a9104dd93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:21.757213 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eab504-57ad-4d43-b2ba-105a9104dd93 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:21.778684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 87/166] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:21 2022] GET /identity/v3/role_assignments?user.id=318df21351594db286347c68d09ec473&scope.project.id=7849898da2494c4897ab5e0e09c9fe9e&role.id=23538dc4c770454e80dd11ed8e498539 => generated 608 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:21.779347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eab504-57ad-4d43-b2ba-105a9104dd93 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:21.821343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3025b51-4f44-4280-bbfb-5405c8c6d392 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:21.844808 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:21.886656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:21.938002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450d5d3-cd39-4bb1-aed6-b557b60aaf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:21.938662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450d5d3-cd39-4bb1-aed6-b557b60aaf95 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:21.938945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450d5d3-cd39-4bb1-aed6-b557b60aaf95 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:21.939218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450d5d3-cd39-4bb1-aed6-b557b60aaf95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:21.972656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450d5d3-cd39-4bb1-aed6-b557b60aaf95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-b450d5d3-cd39-4bb1-aed6-b557b60aaf95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:21.977759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:21.978818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:21.980172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:21.980172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:21.980172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:21.980172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:21.980172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:21.980172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:21.980172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:21.980172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:21.980172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:21.980172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:21.980172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:21.980172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:21.980172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 80/167] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:21.980172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450d5d3-cd39-4bb1-aed6-b557b60aaf95 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:22.050680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eab504-57ad-4d43-b2ba-105a9104dd93 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:22.162846 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4abd0-2397-4b6b-804d-e21c5c280132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:22.163465 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4abd0-2397-4b6b-804d-e21c5c280132 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:22.163782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4abd0-2397-4b6b-804d-e21c5c280132 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:22.164053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4abd0-2397-4b6b-804d-e21c5c280132 None admin] PATH_INFO: `/v3/projects/dae378bdfac34f76b6cc36e4b2fbae94` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:22.168383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4abd0-2397-4b6b-804d-e21c5c280132 None admin] RBAC: Authorizing `identity:get_project(project_id=dae378bdfac34f76b6cc36e4b2fbae94)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:22.169743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450d5d3-cd39-4bb1-aed6-b557b60aaf95 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:22.169854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4abd0-2397-4b6b-804d-e21c5c280132 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:22.171595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 88/168] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:21 2022] GET /identity/v3/projects/dae378bdfac34f76b6cc36e4b2fbae94 => generated 312 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:22.172407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4abd0-2397-4b6b-804d-e21c5c280132 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:22.222517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a995cc-5c76-457a-aac1-8c52070e57f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:22.223500 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a995cc-5c76-457a-aac1-8c52070e57f2 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:22.223810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a995cc-5c76-457a-aac1-8c52070e57f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:22.224087 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a995cc-5c76-457a-aac1-8c52070e57f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:22.241832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a995cc-5c76-457a-aac1-8c52070e57f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-77a995cc-5c76-457a-aac1-8c52070e57f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:22.247810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:22.248689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:22.249864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:22.249864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:22.249864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:22.249864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:22.249864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:22.249864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:22.249864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:22.249864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:22.249864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:22.249864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:22.249864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:22.249864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:22.249864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 81/169] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:14:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:22.249864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a995cc-5c76-457a-aac1-8c52070e57f2 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:22.332965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:22.403835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:22.420298 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4abd0-2397-4b6b-804d-e21c5c280132 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:22.448404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a995cc-5c76-457a-aac1-8c52070e57f2 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:22.497377 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7116100-e582-4d1e-8f05-08dd59d0503f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:22.498167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7116100-e582-4d1e-8f05-08dd59d0503f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:22.498456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7116100-e582-4d1e-8f05-08dd59d0503f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:22.498733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7116100-e582-4d1e-8f05-08dd59d0503f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:22.499470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7116100-e582-4d1e-8f05-08dd59d0503f None admin] RBAC: Adding query filter params (role.id=ebbe7434d1ec4bc4ba71737f55987dd9, scope.project.id=dae378bdfac34f76b6cc36e4b2fbae94, user.id=28ab6d08782b4c26b0afe016bbcd8688) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:22.499806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7116100-e582-4d1e-8f05-08dd59d0503f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:22.501280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7116100-e582-4d1e-8f05-08dd59d0503f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:22.511656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 82/170] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:22 2022] GET /identity/v3/role_assignments?user.id=28ab6d08782b4c26b0afe016bbcd8688&scope.project.id=dae378bdfac34f76b6cc36e4b2fbae94&role.id=ebbe7434d1ec4bc4ba71737f55987dd9 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:22.512579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7116100-e582-4d1e-8f05-08dd59d0503f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:22.517840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17527e-ba00-43dc-b266-323b2a5b712c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:22.518549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17527e-ba00-43dc-b266-323b2a5b712c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:22.518832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17527e-ba00-43dc-b266-323b2a5b712c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:22.519106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17527e-ba00-43dc-b266-323b2a5b712c None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:22.519831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17527e-ba00-43dc-b266-323b2a5b712c None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:22.520136 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17527e-ba00-43dc-b266-323b2a5b712c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:22.521572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17527e-ba00-43dc-b266-323b2a5b712c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:22.538083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 89/171] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:22.548662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17527e-ba00-43dc-b266-323b2a5b712c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:22.683650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7116100-e582-4d1e-8f05-08dd59d0503f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:22.699964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:22.748844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4985d-8337-4711-99ad-bfbc8509470d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:22.749910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4985d-8337-4711-99ad-bfbc8509470d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:22.750413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4985d-8337-4711-99ad-bfbc8509470d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:22.750897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4985d-8337-4711-99ad-bfbc8509470d None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:22.751915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4985d-8337-4711-99ad-bfbc8509470d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:22.752419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4985d-8337-4711-99ad-bfbc8509470d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:22.754134 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4985d-8337-4711-99ad-bfbc8509470d None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:22.763226 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17527e-ba00-43dc-b266-323b2a5b712c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:22.771347 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 83/172] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:14:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:22.776602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4985d-8337-4711-99ad-bfbc8509470d None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:22.832223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6bc1d-1c3d-4daa-ba3c-49d17542c8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:22.832948 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6bc1d-1c3d-4daa-ba3c-49d17542c8dd None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:22.833230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6bc1d-1c3d-4daa-ba3c-49d17542c8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:22.833555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6bc1d-1c3d-4daa-ba3c-49d17542c8dd None admin] PATH_INFO: `/v3/users/28ab6d08782b4c26b0afe016bbcd8688` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:22.834858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6bc1d-1c3d-4daa-ba3c-49d17542c8dd None admin] RBAC: Authorizing `identity:get_user(user_id=28ab6d08782b4c26b0afe016bbcd8688)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:22.836337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6bc1d-1c3d-4daa-ba3c-49d17542c8dd None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:22.838045 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 90/173] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:22 2022] GET /identity/v3/users/28ab6d08782b4c26b0afe016bbcd8688 => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:22.852117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6bc1d-1c3d-4daa-ba3c-49d17542c8dd None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:22.892232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:22.957046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4985d-8337-4711-99ad-bfbc8509470d None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:23.031808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203dbec7-343b-4818-bfd0-662d7e999206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:23.033832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203dbec7-343b-4818-bfd0-662d7e999206 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:23.034496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203dbec7-343b-4818-bfd0-662d7e999206 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:23.036206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203dbec7-343b-4818-bfd0-662d7e999206 None admin] PATH_INFO: `/v3/users/28ab6d08782b4c26b0afe016bbcd8688` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:23.038671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203dbec7-343b-4818-bfd0-662d7e999206 None admin] RBAC: Authorizing `identity:get_user(user_id=28ab6d08782b4c26b0afe016bbcd8688)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:23.040558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203dbec7-343b-4818-bfd0-662d7e999206 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:23.042752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 84/174] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:22 2022] GET /identity/v3/users/28ab6d08782b4c26b0afe016bbcd8688 => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:23.048780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203dbec7-343b-4818-bfd0-662d7e999206 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:23.050861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6bc1d-1c3d-4daa-ba3c-49d17542c8dd None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:23.114753 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:23.125456 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5bdbf9-ff0f-4cc3-9e98-8b24bc53b887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:23.126508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bdbf9-ff0f-4cc3-9e98-8b24bc53b887 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:23.127893 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bdbf9-ff0f-4cc3-9e98-8b24bc53b887 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:23.129059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bdbf9-ff0f-4cc3-9e98-8b24bc53b887 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:23.214055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203dbec7-343b-4818-bfd0-662d7e999206 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:23.243159 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5bdbf9-ff0f-4cc3-9e98-8b24bc53b887 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:23.245084 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5bdbf9-ff0f-4cc3-9e98-8b24bc53b887 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:23.263380 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a5fbf-403d-4c63-ad8c-c16762e98154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:23.264390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a5fbf-403d-4c63-ad8c-c16762e98154 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:23.264983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a5fbf-403d-4c63-ad8c-c16762e98154 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:23.265465 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a5fbf-403d-4c63-ad8c-c16762e98154 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:23.277633 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:23.348524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a5fbf-403d-4c63-ad8c-c16762e98154 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:23.349992 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a5fbf-403d-4c63-ad8c-c16762e98154 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:23.366783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 91/175] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:14:23.366783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d8c9c-6fbd-4a63-aa4d-d2600fef66a3 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:23.366783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d8c9c-6fbd-4a63-aa4d-d2600fef66a3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:23.366783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d8c9c-6fbd-4a63-aa4d-d2600fef66a3 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:23.366783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 92/176] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:23.367697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816f98-b61d-4948-8fbe-14fb442e947e None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:23.367911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816f98-b61d-4948-8fbe-14fb442e947e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:23.368255 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816f98-b61d-4948-8fbe-14fb442e947e None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:23.369683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 93/177] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:23.372746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd467caa-660e-4910-808c-0ebc6f037e09 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:23.373409 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd467caa-660e-4910-808c-0ebc6f037e09 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:23.373687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd467caa-660e-4910-808c-0ebc6f037e09 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:23.375096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 94/178] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:23.379303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd467caa-660e-4910-808c-0ebc6f037e09 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:23.397039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:23.459478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 85/179] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:23.465666 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2e32c-be4f-42c1-95e3-5876683a3210 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:23.465904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2e32c-be4f-42c1-95e3-5876683a3210 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:23.466176 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2e32c-be4f-42c1-95e3-5876683a3210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:23.487600 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-9dc2e32c-be4f-42c1-95e3-5876683a3210 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:23.488846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9dc2e32c-be4f-42c1-95e3-5876683a3210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:23.498794 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd467caa-660e-4910-808c-0ebc6f037e09 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:23.526552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:23.537887 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf240c98-7933-431f-9f18-594522d6b6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:23.538548 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf240c98-7933-431f-9f18-594522d6b6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:23.538855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf240c98-7933-431f-9f18-594522d6b6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:23.539102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf240c98-7933-431f-9f18-594522d6b6e0 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:23.540679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf240c98-7933-431f-9f18-594522d6b6e0 None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:23.542249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf240c98-7933-431f-9f18-594522d6b6e0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:23.544221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 95/180] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:23 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:23.545292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef4221-3c87-4a81-bb23-a2d5ec7f5b03 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:23.545576 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef4221-3c87-4a81-bb23-a2d5ec7f5b03 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:23.545853 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef4221-3c87-4a81-bb23-a2d5ec7f5b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:23.565351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f7ef4221-3c87-4a81-bb23-a2d5ec7f5b03 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:23.566587 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f7ef4221-3c87-4a81-bb23-a2d5ec7f5b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:23.571689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 86/181] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:23.573588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8d797-511c-454a-b292-da5fc30e4c65 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:23.573868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8d797-511c-454a-b292-da5fc30e4c65 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:23.574140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8d797-511c-454a-b292-da5fc30e4c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:23.594461 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-a2c8d797-511c-454a-b292-da5fc30e4c65 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:23.595841 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a2c8d797-511c-454a-b292-da5fc30e4c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:23.665969 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 96/182] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:23.666630 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef4221-3c87-4a81-bb23-a2d5ec7f5b03 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:23.675188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:23.681026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 87/183] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:23.685532 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8d797-511c-454a-b292-da5fc30e4c65 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:23.802651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef4221-3c87-4a81-bb23-a2d5ec7f5b03 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:23.833266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:23.847482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029a824-c8af-4d84-9aa9-1a3e2bf782ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:23.850698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029a824-c8af-4d84-9aa9-1a3e2bf782ac None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:23.851148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029a824-c8af-4d84-9aa9-1a3e2bf782ac None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:23.851587 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029a824-c8af-4d84-9aa9-1a3e2bf782ac None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:23.854427 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029a824-c8af-4d84-9aa9-1a3e2bf782ac None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:23.856239 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029a824-c8af-4d84-9aa9-1a3e2bf782ac None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:23.858176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 97/184] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:23 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:23.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e54f5-f619-43f9-882a-e64bec163dc1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:23.859931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e54f5-f619-43f9-882a-e64bec163dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:23.860366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e54f5-f619-43f9-882a-e64bec163dc1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:23.861715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 98/185] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:23.862565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e54f5-f619-43f9-882a-e64bec163dc1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:23.870687 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8d797-511c-454a-b292-da5fc30e4c65 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:23.906094 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fcb31-573a-471c-94a7-b35e4581ab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:23.906935 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fcb31-573a-471c-94a7-b35e4581ab66 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:23.907280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fcb31-573a-471c-94a7-b35e4581ab66 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:23.907961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fcb31-573a-471c-94a7-b35e4581ab66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:23.908750 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fcb31-573a-471c-94a7-b35e4581ab66 None admin] RBAC: Adding query filter params (role.id=ebbe7434d1ec4bc4ba71737f55987dd9, scope.project.id=7849898da2494c4897ab5e0e09c9fe9e, user.id=28ab6d08782b4c26b0afe016bbcd8688) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:23.909120 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fcb31-573a-471c-94a7-b35e4581ab66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:23.921082 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fcb31-573a-471c-94a7-b35e4581ab66 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:23.927670 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 88/186] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:23 2022] GET /identity/v3/role_assignments?user.id=28ab6d08782b4c26b0afe016bbcd8688&scope.project.id=7849898da2494c4897ab5e0e09c9fe9e&role.id=ebbe7434d1ec4bc4ba71737f55987dd9 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:23.928540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fcb31-573a-471c-94a7-b35e4581ab66 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:23.998433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e54f5-f619-43f9-882a-e64bec163dc1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:24.019042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:24.037340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00636c61-d38f-434c-8124-519a899800d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:24.038000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00636c61-d38f-434c-8124-519a899800d5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:24.038279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00636c61-d38f-434c-8124-519a899800d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:24.038555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00636c61-d38f-434c-8124-519a899800d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:24.050694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00636c61-d38f-434c-8124-519a899800d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-00636c61-d38f-434c-8124-519a899800d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:24.055829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:24.056807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:24.057588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:24.057588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:24.057588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:24.057588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:24.057588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:24.057588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:24.057588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:24.057588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:24.057588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:24.057588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:24.057588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:24.057588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:24.057588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 99/187] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:14:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:24.058243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3acfa-ae7e-4912-ae80-5dbd23093ca5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:24.063828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3acfa-ae7e-4912-ae80-5dbd23093ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:24.063828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3acfa-ae7e-4912-ae80-5dbd23093ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:24.068912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fcb31-573a-471c-94a7-b35e4581ab66 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:24.078544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-94e3acfa-ae7e-4912-ae80-5dbd23093ca5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:24.079830 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-94e3acfa-ae7e-4912-ae80-5dbd23093ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:24.108535 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81001bcd-ebf9-412f-8cbd-17373fa7c623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:24.109511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81001bcd-ebf9-412f-8cbd-17373fa7c623 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:24.109818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81001bcd-ebf9-412f-8cbd-17373fa7c623 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:24.110161 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81001bcd-ebf9-412f-8cbd-17373fa7c623 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:24.111145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81001bcd-ebf9-412f-8cbd-17373fa7c623 None admin] RBAC: Adding query filter params (role.id=31058b9633b44d7ca64f103119987fd9, scope.project.id=7849898da2494c4897ab5e0e09c9fe9e, user.id=28ab6d08782b4c26b0afe016bbcd8688) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:24.111734 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81001bcd-ebf9-412f-8cbd-17373fa7c623 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:24.113497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81001bcd-ebf9-412f-8cbd-17373fa7c623 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:24.119961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 89/188] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:23 2022] GET /identity/v3/role_assignments?user.id=28ab6d08782b4c26b0afe016bbcd8688&scope.project.id=7849898da2494c4897ab5e0e09c9fe9e&role.id=31058b9633b44d7ca64f103119987fd9 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:24.120829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81001bcd-ebf9-412f-8cbd-17373fa7c623 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:24.134024 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:24.172751 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 100/189] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:24.174273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3acfa-ae7e-4912-ae80-5dbd23093ca5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:24.248357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81001bcd-ebf9-412f-8cbd-17373fa7c623 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:24.288072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:24.292276 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925dd04-495b-49df-a36c-18c081e0bc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:24.293628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925dd04-495b-49df-a36c-18c081e0bc3c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:24.294092 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925dd04-495b-49df-a36c-18c081e0bc3c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:24.294538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925dd04-495b-49df-a36c-18c081e0bc3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:24.300124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3acfa-ae7e-4912-ae80-5dbd23093ca5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:24.310448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925dd04-495b-49df-a36c-18c081e0bc3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-d925dd04-495b-49df-a36c-18c081e0bc3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:24.316592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:24.317534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:24.318308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:24.318308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:24.318308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:24.318308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:24.318308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:24.318308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:24.318308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:24.318308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:24.318308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:24.318308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:24.318308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:24.318308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:24.320592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 90/190] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:14:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:24.321882 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925dd04-495b-49df-a36c-18c081e0bc3c None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:24.344834 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130cc8c-75e5-408d-84a2-832ecfe46f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:24.344834 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130cc8c-75e5-408d-84a2-832ecfe46f68 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:24.344834 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130cc8c-75e5-408d-84a2-832ecfe46f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:24.345144 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130cc8c-75e5-408d-84a2-832ecfe46f68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:24.346946 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130cc8c-75e5-408d-84a2-832ecfe46f68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:24.348485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130cc8c-75e5-408d-84a2-832ecfe46f68 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:24.351684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 101/191] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:14:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:24.352717 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130cc8c-75e5-408d-84a2-832ecfe46f68 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:24.417555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:24.458149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925dd04-495b-49df-a36c-18c081e0bc3c None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:24.478992 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130cc8c-75e5-408d-84a2-832ecfe46f68 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:24.491809 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12546f6a-d378-48ed-af02-5ed8f56814a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:24.492458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12546f6a-d378-48ed-af02-5ed8f56814a4 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:24.492746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12546f6a-d378-48ed-af02-5ed8f56814a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:24.493016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12546f6a-d378-48ed-af02-5ed8f56814a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:24.493668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12546f6a-d378-48ed-af02-5ed8f56814a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:24.493968 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12546f6a-d378-48ed-af02-5ed8f56814a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:24.495914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12546f6a-d378-48ed-af02-5ed8f56814a4 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:24.503664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 91/192] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:24.504525 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12546f6a-d378-48ed-af02-5ed8f56814a4 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:24.514280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af13255-94d6-4a13-843c-ab20316c664f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:24.514888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af13255-94d6-4a13-843c-ab20316c664f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:24.515171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af13255-94d6-4a13-843c-ab20316c664f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:24.515448 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af13255-94d6-4a13-843c-ab20316c664f None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:24.516145 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af13255-94d6-4a13-843c-ab20316c664f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:24.516451 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af13255-94d6-4a13-843c-ab20316c664f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:24.517814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af13255-94d6-4a13-843c-ab20316c664f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:24.525988 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 102/193] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:14:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:24.526782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af13255-94d6-4a13-843c-ab20316c664f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:24.557557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:24.607627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12546f6a-d378-48ed-af02-5ed8f56814a4 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:24.626685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af13255-94d6-4a13-843c-ab20316c664f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:24.635183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:24.640540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ea4ea-7ed0-4fd3-918c-1c6e561ed689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:24.641181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ea4ea-7ed0-4fd3-918c-1c6e561ed689 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:24.641469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ea4ea-7ed0-4fd3-918c-1c6e561ed689 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:24.641746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ea4ea-7ed0-4fd3-918c-1c6e561ed689 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:24.642394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ea4ea-7ed0-4fd3-918c-1c6e561ed689 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:24.643877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ea4ea-7ed0-4fd3-918c-1c6e561ed689 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:24.660206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe94c8-f40a-4ea5-b168-7e58ba951b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:24.660833 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe94c8-f40a-4ea5-b168-7e58ba951b0e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:24.661117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe94c8-f40a-4ea5-b168-7e58ba951b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:24.661391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe94c8-f40a-4ea5-b168-7e58ba951b0e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:24.664107 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 92/194] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:14:24 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:14:24.664910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ea4ea-7ed0-4fd3-918c-1c6e561ed689 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:24.674863 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe94c8-f40a-4ea5-b168-7e58ba951b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-3cbe94c8-f40a-4ea5-b168-7e58ba951b0e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:24.682804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:24.684193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:24.684998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:24.684998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:24.684998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:24.684998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:24.684998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:24.684998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:24.684998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:24.684998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:24.684998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:24.684998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:24.684998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:24.684998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:24.684998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 103/195] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:24.686399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe94c8-f40a-4ea5-b168-7e58ba951b0e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:24.793339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ea4ea-7ed0-4fd3-918c-1c6e561ed689 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:24.799657 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:24.808043 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe94c8-f40a-4ea5-b168-7e58ba951b0e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:24.843938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26a189-4a39-4629-9845-f82f8ba6ba1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:24.844581 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26a189-4a39-4629-9845-f82f8ba6ba1a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:24.844862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26a189-4a39-4629-9845-f82f8ba6ba1a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:24.845141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26a189-4a39-4629-9845-f82f8ba6ba1a None admin] PATH_INFO: `/v3/users/0bfe2bebfbd4476fabb2bac97e8aa9d4` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:24.852886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3424d-852d-4d2d-97fe-69c5061d504e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:24.854036 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3424d-852d-4d2d-97fe-69c5061d504e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:24.854323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3424d-852d-4d2d-97fe-69c5061d504e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:24.854984 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3424d-852d-4d2d-97fe-69c5061d504e None admin] PATH_INFO: `/v3/users/0bfe2bebfbd4476fabb2bac97e8aa9d4` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:24.863158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26a189-4a39-4629-9845-f82f8ba6ba1a None admin] RBAC: Authorizing `identity:get_user(user_id=0bfe2bebfbd4476fabb2bac97e8aa9d4)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:24.865106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26a189-4a39-4629-9845-f82f8ba6ba1a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:24.866864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 93/196] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:24 2022] GET /identity/v3/users/0bfe2bebfbd4476fabb2bac97e8aa9d4 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:24.867159 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26a189-4a39-4629-9845-f82f8ba6ba1a None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:24.874123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3424d-852d-4d2d-97fe-69c5061d504e None admin] RBAC: Authorizing `identity:get_user(user_id=0bfe2bebfbd4476fabb2bac97e8aa9d4)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:24.875819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3424d-852d-4d2d-97fe-69c5061d504e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:24.877478 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 104/197] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:24 2022] GET /identity/v3/users/0bfe2bebfbd4476fabb2bac97e8aa9d4 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:24.878301 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3424d-852d-4d2d-97fe-69c5061d504e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:24.913273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:24.997259 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26a189-4a39-4629-9845-f82f8ba6ba1a None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:25.006735 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3424d-852d-4d2d-97fe-69c5061d504e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:25.041467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b9115-e548-454b-8e92-8888887fa35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:25.042325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b9115-e548-454b-8e92-8888887fa35a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:25.042602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b9115-e548-454b-8e92-8888887fa35a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:25.042868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b9115-e548-454b-8e92-8888887fa35a None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:25.043530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b9115-e548-454b-8e92-8888887fa35a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:25.043925 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b9115-e548-454b-8e92-8888887fa35a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:25.045234 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b9115-e548-454b-8e92-8888887fa35a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:25.046907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2bc3d-6297-4dbc-b737-74b410b11ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:25.047646 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2bc3d-6297-4dbc-b737-74b410b11ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:25.048025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2bc3d-6297-4dbc-b737-74b410b11ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:25.048397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2bc3d-6297-4dbc-b737-74b410b11ca4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:25.055473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 94/198] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:25.056372 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b9115-e548-454b-8e92-8888887fa35a None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:25.085075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:25.148306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2bc3d-6297-4dbc-b737-74b410b11ca4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:25.149813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2bc3d-6297-4dbc-b737-74b410b11ca4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:25.184999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:25.187382 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b9115-e548-454b-8e92-8888887fa35a None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:25.224811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13391b6d-29d9-4f47-9f29-c581da9549e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:25.225790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13391b6d-29d9-4f47-9f29-c581da9549e9 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:25.226232 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13391b6d-29d9-4f47-9f29-c581da9549e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:25.226650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13391b6d-29d9-4f47-9f29-c581da9549e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:25.275837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 105/199] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:25.275837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2bc3d-6297-4dbc-b737-74b410b11ca4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:25.302531 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13391b6d-29d9-4f47-9f29-c581da9549e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:25.304052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13391b6d-29d9-4f47-9f29-c581da9549e9 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:25.329708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:25.389985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2bc3d-6297-4dbc-b737-74b410b11ca4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:25.422708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 95/200] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:25.424013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13391b6d-29d9-4f47-9f29-c581da9549e9 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:25.431329 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8be06c-dcfc-4b16-9e14-21ba8f97b499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:25.432143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8be06c-dcfc-4b16-9e14-21ba8f97b499 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:25.432489 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8be06c-dcfc-4b16-9e14-21ba8f97b499 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:25.432832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8be06c-dcfc-4b16-9e14-21ba8f97b499 None admin] PATH_INFO: `/v3/users/c2f0f8565e6445ce867bb3ea06d9b867` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:25.434526 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8be06c-dcfc-4b16-9e14-21ba8f97b499 None admin] RBAC: Authorizing `identity:get_user(user_id=c2f0f8565e6445ce867bb3ea06d9b867)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:25.436559 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8be06c-dcfc-4b16-9e14-21ba8f97b499 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:25.438944 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 106/201] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:25 2022] GET /identity/v3/users/c2f0f8565e6445ce867bb3ea06d9b867 => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:25.440011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8be06c-dcfc-4b16-9e14-21ba8f97b499 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:25.467954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:25.558138 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13391b6d-29d9-4f47-9f29-c581da9549e9 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:25.562418 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8be06c-dcfc-4b16-9e14-21ba8f97b499 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:25.590973 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:25.601269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2bd13-aed7-4185-b8e1-512b3f9b24d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:25.602246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2bd13-aed7-4185-b8e1-512b3f9b24d0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:25.602693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2bd13-aed7-4185-b8e1-512b3f9b24d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:25.603149 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2bd13-aed7-4185-b8e1-512b3f9b24d0 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:25.604640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2bd13-aed7-4185-b8e1-512b3f9b24d0 None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:25.606474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42c6b8-7e16-444c-b87b-22b98d57e37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:25.606593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2bd13-aed7-4185-b8e1-512b3f9b24d0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:25.607398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42c6b8-7e16-444c-b87b-22b98d57e37d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:25.607967 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42c6b8-7e16-444c-b87b-22b98d57e37d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:25.608410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42c6b8-7e16-444c-b87b-22b98d57e37d None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:25.608784 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 107/202] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:25 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:25.609920 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2bd13-aed7-4185-b8e1-512b3f9b24d0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:25.610134 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42c6b8-7e16-444c-b87b-22b98d57e37d None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:25.611865 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42c6b8-7e16-444c-b87b-22b98d57e37d None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:25.615119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 96/203] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:25 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:25.616188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42c6b8-7e16-444c-b87b-22b98d57e37d None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:25.682787 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:25.729029 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2bd13-aed7-4185-b8e1-512b3f9b24d0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:25.752835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42c6b8-7e16-444c-b87b-22b98d57e37d None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:25.772951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e862e0-fcbc-49f5-ba99-72d6f18f6b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:25.773944 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e862e0-fcbc-49f5-ba99-72d6f18f6b12 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:25.774436 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e862e0-fcbc-49f5-ba99-72d6f18f6b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:25.774898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e862e0-fcbc-49f5-ba99-72d6f18f6b12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:25.792279 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f049db-d5b0-4dfe-a7e0-a273e8ca9f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:25.792914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f049db-d5b0-4dfe-a7e0-a273e8ca9f0a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:25.793172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f049db-d5b0-4dfe-a7e0-a273e8ca9f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:25.793444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f049db-d5b0-4dfe-a7e0-a273e8ca9f0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:25.794172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f049db-d5b0-4dfe-a7e0-a273e8ca9f0a None admin] RBAC: Adding query filter params (role.id=31058b9633b44d7ca64f103119987fd9, scope.project.id=83919a658b6446d098540ce590417c23, user.id=0bfe2bebfbd4476fabb2bac97e8aa9d4) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:25.794480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f049db-d5b0-4dfe-a7e0-a273e8ca9f0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:25.795987 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f049db-d5b0-4dfe-a7e0-a273e8ca9f0a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:25.801973 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 97/204] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:25 2022] GET /identity/v3/role_assignments?user.id=0bfe2bebfbd4476fabb2bac97e8aa9d4&scope.project.id=83919a658b6446d098540ce590417c23&role.id=31058b9633b44d7ca64f103119987fd9 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:25.802903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f049db-d5b0-4dfe-a7e0-a273e8ca9f0a None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:25.864504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e862e0-fcbc-49f5-ba99-72d6f18f6b12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:25.866951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e862e0-fcbc-49f5-ba99-72d6f18f6b12 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:25.909411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:25.966767 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f049db-d5b0-4dfe-a7e0-a273e8ca9f0a None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:26.008849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa26b3-56a7-4ff3-884b-5cacf8105e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:26.011002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa26b3-56a7-4ff3-884b-5cacf8105e1e None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:26.011648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa26b3-56a7-4ff3-884b-5cacf8105e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:26.012401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa26b3-56a7-4ff3-884b-5cacf8105e1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:26.013381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:26.014706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa26b3-56a7-4ff3-884b-5cacf8105e1e None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=83919a658b6446d098540ce590417c23, user.id=0bfe2bebfbd4476fabb2bac97e8aa9d4) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:26.015283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa26b3-56a7-4ff3-884b-5cacf8105e1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:26.017591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa26b3-56a7-4ff3-884b-5cacf8105e1e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:26.026057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 98/205] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:25 2022] GET /identity/v3/role_assignments?user.id=0bfe2bebfbd4476fabb2bac97e8aa9d4&scope.project.id=83919a658b6446d098540ce590417c23&role.id=23538dc4c770454e80dd11ed8e498539 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:26.067323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 108/206] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:26.073599 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa26b3-56a7-4ff3-884b-5cacf8105e1e None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:26.225347 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa26b3-56a7-4ff3-884b-5cacf8105e1e None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:26.254335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:26.273814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aaba08-1d17-4f15-999c-d73959843410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:26.274453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaba08-1d17-4f15-999c-d73959843410 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:26.274737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaba08-1d17-4f15-999c-d73959843410 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:26.275012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaba08-1d17-4f15-999c-d73959843410 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:26.280264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aaba08-1d17-4f15-999c-d73959843410 None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:26.281741 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aaba08-1d17-4f15-999c-d73959843410 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:26.283682 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 99/207] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:26 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:26.289540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e862e0-fcbc-49f5-ba99-72d6f18f6b12 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:26.398258 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e862e0-fcbc-49f5-ba99-72d6f18f6b12 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:26.439048 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b1da3-e4a5-4f30-94af-27387f9a3809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:26.439689 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b1da3-e4a5-4f30-94af-27387f9a3809 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:26.439978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b1da3-e4a5-4f30-94af-27387f9a3809 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:26.440257 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b1da3-e4a5-4f30-94af-27387f9a3809 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:26.440987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b1da3-e4a5-4f30-94af-27387f9a3809 None admin] RBAC: Adding query filter params (role.id=c6bc2b1713e84fecb6708753f6ba6a66, scope.project.id=7849898da2494c4897ab5e0e09c9fe9e, user.id=c2f0f8565e6445ce867bb3ea06d9b867) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:26.441300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b1da3-e4a5-4f30-94af-27387f9a3809 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:26.442734 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b1da3-e4a5-4f30-94af-27387f9a3809 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:26.449681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 109/208] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:26 2022] GET /identity/v3/role_assignments?user.id=c2f0f8565e6445ce867bb3ea06d9b867&scope.project.id=7849898da2494c4897ab5e0e09c9fe9e&role.id=c6bc2b1713e84fecb6708753f6ba6a66 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:27.284369 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7526495-575f-4278-b5fe-f381fab0e593 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:27.285368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7526495-575f-4278-b5fe-f381fab0e593 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:27.285871 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7526495-575f-4278-b5fe-f381fab0e593 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:27.287733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 100/209] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:27.293822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b2a2b-eac9-41b9-bfc0-1796866e81ae None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:27.294674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b2a2b-eac9-41b9-bfc0-1796866e81ae None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:27.295149 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b2a2b-eac9-41b9-bfc0-1796866e81ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:27.313947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0e6b2a2b-eac9-41b9-bfc0-1796866e81ae None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:27.315454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0e6b2a2b-eac9-41b9-bfc0-1796866e81ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:27.337262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:27.354231 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66aedcc-167e-47d1-be9b-dec61111a571 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:27.354536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66aedcc-167e-47d1-be9b-dec61111a571 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:27.354818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66aedcc-167e-47d1-be9b-dec61111a571 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:27.357109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 101/210] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:27.362827 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd885cf-b4b2-4e2a-97be-583f9131186f None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:27.363114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd885cf-b4b2-4e2a-97be-583f9131186f None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:27.363396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd885cf-b4b2-4e2a-97be-583f9131186f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:27.386382 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-9bd885cf-b4b2-4e2a-97be-583f9131186f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:27.388127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9bd885cf-b4b2-4e2a-97be-583f9131186f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:27.392400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 110/211] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:27.458851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 102/212] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:27.676204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b2a2b-eac9-41b9-bfc0-1796866e81ae None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:27.772828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd885cf-b4b2-4e2a-97be-583f9131186f None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:27.787394 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b2a2b-eac9-41b9-bfc0-1796866e81ae None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:27.826916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92c021-1b2d-46a6-b3d8-231c9809901f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:27.827927 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92c021-1b2d-46a6-b3d8-231c9809901f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:27.828406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92c021-1b2d-46a6-b3d8-231c9809901f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:27.828893 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92c021-1b2d-46a6-b3d8-231c9809901f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:27.840288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92c021-1b2d-46a6-b3d8-231c9809901f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-cb92c021-1b2d-46a6-b3d8-231c9809901f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:27.845498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:27.846372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:27.847184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:27.847184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:27.847184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:27.847184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:27.847184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:27.847184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:27.847184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:27.847184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:27.847184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:27.847184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:27.847184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:27.847184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:27.849497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 111/213] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:27.856104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92c021-1b2d-46a6-b3d8-231c9809901f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:27.877444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:27.954427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd885cf-b4b2-4e2a-97be-583f9131186f None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:27.986153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:27.995600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92c021-1b2d-46a6-b3d8-231c9809901f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:28.017803 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32971738-61c6-406e-9aae-35eed91e424b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:28.018810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32971738-61c6-406e-9aae-35eed91e424b None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:28.019342 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32971738-61c6-406e-9aae-35eed91e424b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:28.019802 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32971738-61c6-406e-9aae-35eed91e424b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:28.021693 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32971738-61c6-406e-9aae-35eed91e424b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:28.024413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32971738-61c6-406e-9aae-35eed91e424b None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:28.031879 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 103/214] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:14:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:28.034606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae94b5b-d9d8-47cc-b162-3b3af2526641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:28.035738 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94b5b-d9d8-47cc-b162-3b3af2526641 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:28.036215 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94b5b-d9d8-47cc-b162-3b3af2526641 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:28.036701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94b5b-d9d8-47cc-b162-3b3af2526641 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:28.036803 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32971738-61c6-406e-9aae-35eed91e424b None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:28.037753 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae94b5b-d9d8-47cc-b162-3b3af2526641 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:28.038268 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae94b5b-d9d8-47cc-b162-3b3af2526641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:28.039892 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae94b5b-d9d8-47cc-b162-3b3af2526641 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:28.054088 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 112/215] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:28.055545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae94b5b-d9d8-47cc-b162-3b3af2526641 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:28.142225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:28.166800 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae94b5b-d9d8-47cc-b162-3b3af2526641 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:28.203447 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfec84-73b9-4e11-8fad-742daf4d9285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:28.204311 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfec84-73b9-4e11-8fad-742daf4d9285 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:28.204759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfec84-73b9-4e11-8fad-742daf4d9285 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:28.205287 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfec84-73b9-4e11-8fad-742daf4d9285 None admin] PATH_INFO: `/v3/users/9500c1cf0d9f4b4c96ad281fbe5245e5` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:28.219911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abfec84-73b9-4e11-8fad-742daf4d9285 None admin] RBAC: Authorizing `identity:get_user(user_id=9500c1cf0d9f4b4c96ad281fbe5245e5)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:28.220406 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32971738-61c6-406e-9aae-35eed91e424b None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:28.221681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abfec84-73b9-4e11-8fad-742daf4d9285 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:28.223464 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 113/216] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:28 2022] GET /identity/v3/users/9500c1cf0d9f4b4c96ad281fbe5245e5 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:28.229639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94f3d0-1a91-444f-a27f-c8d8b221780a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:28.229921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94f3d0-1a91-444f-a27f-c8d8b221780a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:28.230193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94f3d0-1a91-444f-a27f-c8d8b221780a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:28.231668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 114/217] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:28.236723 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94f3d0-1a91-444f-a27f-c8d8b221780a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:28.265404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70126f3-e6f3-4bce-9bc9-a93f38cd1ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:28.266413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70126f3-e6f3-4bce-9bc9-a93f38cd1ec6 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:28.266895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70126f3-e6f3-4bce-9bc9-a93f38cd1ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:28.267344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70126f3-e6f3-4bce-9bc9-a93f38cd1ec6 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:28.268222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70126f3-e6f3-4bce-9bc9-a93f38cd1ec6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:28.270071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70126f3-e6f3-4bce-9bc9-a93f38cd1ec6 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:28.296857 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 104/218] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:14:28 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:14:28.298960 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc21626-73b9-47c0-886d-fad542413d38 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:28.299390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc21626-73b9-47c0-886d-fad542413d38 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:28.299868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc21626-73b9-47c0-886d-fad542413d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:28.332706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-5dc21626-73b9-47c0-886d-fad542413d38 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:28.335019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5dc21626-73b9-47c0-886d-fad542413d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:28.371333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94f3d0-1a91-444f-a27f-c8d8b221780a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:28.394870 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:28.420667 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d79adf-e20e-4b56-849a-137598ebf2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:28.421577 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d79adf-e20e-4b56-849a-137598ebf2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:28.422002 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d79adf-e20e-4b56-849a-137598ebf2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:28.422418 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d79adf-e20e-4b56-849a-137598ebf2b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:28.426576 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:28.442210 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 105/219] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:28.495771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d79adf-e20e-4b56-849a-137598ebf2b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:28.497143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d79adf-e20e-4b56-849a-137598ebf2b3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:28.604627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 115/220] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:28.609572 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc21626-73b9-47c0-886d-fad542413d38 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:28.693338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d15d5a-9deb-42d8-94e1-06b08c01dded None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:28.693623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d15d5a-9deb-42d8-94e1-06b08c01dded None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:28.693891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d15d5a-9deb-42d8-94e1-06b08c01dded None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:28.695241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 116/221] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:14:28.700317 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b3871-12a3-4710-9f7a-05758f4115b0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:28.700597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b3871-12a3-4710-9f7a-05758f4115b0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:28.700869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b3871-12a3-4710-9f7a-05758f4115b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:28.721271 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d07b3871-12a3-4710-9f7a-05758f4115b0 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:28.722801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d07b3871-12a3-4710-9f7a-05758f4115b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:28.722926 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc21626-73b9-47c0-886d-fad542413d38 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:28.762135 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:28.768121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9660db3-66a2-48c2-af5a-7e0b7753ee33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:28.768826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9660db3-66a2-48c2-af5a-7e0b7753ee33 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:28.769057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9660db3-66a2-48c2-af5a-7e0b7753ee33 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:28.769331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9660db3-66a2-48c2-af5a-7e0b7753ee33 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:28.770476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9660db3-66a2-48c2-af5a-7e0b7753ee33 None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:28.772048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9660db3-66a2-48c2-af5a-7e0b7753ee33 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:28.774011 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 106/222] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:28 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:28.774798 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9660db3-66a2-48c2-af5a-7e0b7753ee33 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:28.805388 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 117/223] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:28.811690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b3871-12a3-4710-9f7a-05758f4115b0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:28.840811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:28.913305 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9660db3-66a2-48c2-af5a-7e0b7753ee33 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:28.956689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c84559-bf4e-42d4-ac18-33aa25e3d929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:28.957349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c84559-bf4e-42d4-ac18-33aa25e3d929 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:28.957698 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c84559-bf4e-42d4-ac18-33aa25e3d929 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:28.957975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c84559-bf4e-42d4-ac18-33aa25e3d929 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:28.968945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c84559-bf4e-42d4-ac18-33aa25e3d929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-a6c84559-bf4e-42d4-ac18-33aa25e3d929 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:28.983965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:28.984916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:28.985708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:28.985708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:28.985708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:28.985708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:28.985708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:28.985708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:28.985708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:28.985708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:28.985708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:28.985708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:28.985708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:28.985708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:28.985708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 107/224] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:14:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:28.992426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c84559-bf4e-42d4-ac18-33aa25e3d929 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:29.029744 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b3871-12a3-4710-9f7a-05758f4115b0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:29.079734 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738fc83-2d69-487d-9fd3-73adaeb1f8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:29.080408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738fc83-2d69-487d-9fd3-73adaeb1f8df None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:29.080685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738fc83-2d69-487d-9fd3-73adaeb1f8df None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:29.081007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738fc83-2d69-487d-9fd3-73adaeb1f8df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:29.081966 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738fc83-2d69-487d-9fd3-73adaeb1f8df None admin] RBAC: Adding query filter params (role.id=ebbe7434d1ec4bc4ba71737f55987dd9, scope.project.id=83919a658b6446d098540ce590417c23, user.id=9500c1cf0d9f4b4c96ad281fbe5245e5) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:29.082277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738fc83-2d69-487d-9fd3-73adaeb1f8df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:29.083822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738fc83-2d69-487d-9fd3-73adaeb1f8df None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:29.090235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 118/225] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:28 2022] GET /identity/v3/role_assignments?user.id=9500c1cf0d9f4b4c96ad281fbe5245e5&scope.project.id=83919a658b6446d098540ce590417c23&role.id=ebbe7434d1ec4bc4ba71737f55987dd9 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:29.119246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738fc83-2d69-487d-9fd3-73adaeb1f8df None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:29.184821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:29.185487 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:29.189954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c84559-bf4e-42d4-ac18-33aa25e3d929 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:29.264842 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d340aba-f9fa-49af-8643-9a0179c54b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:29.265648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d340aba-f9fa-49af-8643-9a0179c54b62 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:29.265929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d340aba-f9fa-49af-8643-9a0179c54b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:29.266774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d340aba-f9fa-49af-8643-9a0179c54b62 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:29.267462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d340aba-f9fa-49af-8643-9a0179c54b62 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:29.267814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d340aba-f9fa-49af-8643-9a0179c54b62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:29.269237 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d340aba-f9fa-49af-8643-9a0179c54b62 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:29.279925 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 108/226] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:14:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:29.295178 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d340aba-f9fa-49af-8643-9a0179c54b62 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:29.315802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738fc83-2d69-487d-9fd3-73adaeb1f8df None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:29.352874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62068461-c9d7-4bc4-b783-6e5663e005d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:29.354259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62068461-c9d7-4bc4-b783-6e5663e005d7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:29.354545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62068461-c9d7-4bc4-b783-6e5663e005d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:29.355265 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62068461-c9d7-4bc4-b783-6e5663e005d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:29.368306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62068461-c9d7-4bc4-b783-6e5663e005d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-62068461-c9d7-4bc4-b783-6e5663e005d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:29.374015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:29.375336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:29.376151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:29.376151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:29.376151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:29.376151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:29.376151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:29.376151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:29.376151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:29.376151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:29.376151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:29.376151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:29.376151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:29.376151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:29.376151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 119/227] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:14:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:14:29.381907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62068461-c9d7-4bc4-b783-6e5663e005d7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:29.468795 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d340aba-f9fa-49af-8643-9a0179c54b62 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:29.485035 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:29.496204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62068461-c9d7-4bc4-b783-6e5663e005d7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:29.533266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b77ea-9ed1-42eb-86b1-1ae269ce3838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:29.533804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b77ea-9ed1-42eb-86b1-1ae269ce3838 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:29.534129 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b77ea-9ed1-42eb-86b1-1ae269ce3838 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:29.534413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b77ea-9ed1-42eb-86b1-1ae269ce3838 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:29.535089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b77ea-9ed1-42eb-86b1-1ae269ce3838 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:29.535393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b77ea-9ed1-42eb-86b1-1ae269ce3838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:29.536820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b77ea-9ed1-42eb-86b1-1ae269ce3838 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:29.546322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 120/228] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:29.547821 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cca85f-5a29-4b65-9880-7a1a8df8268d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:29.548583 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cca85f-5a29-4b65-9880-7a1a8df8268d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:29.548866 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cca85f-5a29-4b65-9880-7a1a8df8268d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:29.549142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cca85f-5a29-4b65-9880-7a1a8df8268d None admin] PATH_INFO: `/v3/users/0bfe2bebfbd4476fabb2bac97e8aa9d4` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:29.550423 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cca85f-5a29-4b65-9880-7a1a8df8268d None admin] RBAC: Authorizing `identity:get_user(user_id=0bfe2bebfbd4476fabb2bac97e8aa9d4)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:29.552051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cca85f-5a29-4b65-9880-7a1a8df8268d None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:29.552256 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b77ea-9ed1-42eb-86b1-1ae269ce3838 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:29.553936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 109/229] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:29 2022] GET /identity/v3/users/0bfe2bebfbd4476fabb2bac97e8aa9d4 => generated 283 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:29.572829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cca85f-5a29-4b65-9880-7a1a8df8268d None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:29.665003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:29.665815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b77ea-9ed1-42eb-86b1-1ae269ce3838 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:29.706577 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c11af7-2ef2-4e68-87f0-ad06d6b4503e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:29.707209 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c11af7-2ef2-4e68-87f0-ad06d6b4503e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:29.707489 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c11af7-2ef2-4e68-87f0-ad06d6b4503e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:29.707809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c11af7-2ef2-4e68-87f0-ad06d6b4503e None admin] PATH_INFO: `/v3/users/0bfe2bebfbd4476fabb2bac97e8aa9d4` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:29.715135 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c11af7-2ef2-4e68-87f0-ad06d6b4503e None admin] RBAC: Authorizing `identity:get_user(user_id=0bfe2bebfbd4476fabb2bac97e8aa9d4)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:29.717340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c11af7-2ef2-4e68-87f0-ad06d6b4503e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:29.719104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 121/230] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:29 2022] GET /identity/v3/users/0bfe2bebfbd4476fabb2bac97e8aa9d4 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:29.725384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c11af7-2ef2-4e68-87f0-ad06d6b4503e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:29.771196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:29.813254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cca85f-5a29-4b65-9880-7a1a8df8268d None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:29.875608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12c459-b7b4-43a9-a2d1-2d99ad63bdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:29.876248 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12c459-b7b4-43a9-a2d1-2d99ad63bdf2 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:29.876529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12c459-b7b4-43a9-a2d1-2d99ad63bdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:29.876803 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12c459-b7b4-43a9-a2d1-2d99ad63bdf2 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:29.877937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12c459-b7b4-43a9-a2d1-2d99ad63bdf2 None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:29.879389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12c459-b7b4-43a9-a2d1-2d99ad63bdf2 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:29.882767 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 110/231] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:29 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:29.887956 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12c459-b7b4-43a9-a2d1-2d99ad63bdf2 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:29.904945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c11af7-2ef2-4e68-87f0-ad06d6b4503e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:29.968185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb924c-b18d-4667-9f1b-fb6a3d89bc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:29.968823 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb924c-b18d-4667-9f1b-fb6a3d89bc9c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:29.969103 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb924c-b18d-4667-9f1b-fb6a3d89bc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:29.969374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb924c-b18d-4667-9f1b-fb6a3d89bc9c None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:29.971250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb924c-b18d-4667-9f1b-fb6a3d89bc9c None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:29.972722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb924c-b18d-4667-9f1b-fb6a3d89bc9c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:29.974365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 122/232] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:29 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:29.980975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb924c-b18d-4667-9f1b-fb6a3d89bc9c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:30.088138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:30.093420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb924c-b18d-4667-9f1b-fb6a3d89bc9c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:30.127904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:30.132669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12c459-b7b4-43a9-a2d1-2d99ad63bdf2 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:30.148309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dd6aa-86ca-4172-9f6b-06038095cce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:30.148977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dd6aa-86ca-4172-9f6b-06038095cce8 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:30.149261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dd6aa-86ca-4172-9f6b-06038095cce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:30.149539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dd6aa-86ca-4172-9f6b-06038095cce8 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23/users/0bfe2bebfbd4476fabb2bac97e8aa9d4/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:30.156350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dd6aa-86ca-4172-9f6b-06038095cce8 None admin] RBAC: Authorizing `identity:create_grant(project_id=83919a658b6446d098540ce590417c23, user_id=0bfe2bebfbd4476fabb2bac97e8aa9d4, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:30.158842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dd6aa-86ca-4172-9f6b-06038095cce8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:30.186413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 123/233] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:14:29 2022] PUT /identity/v3/projects/83919a658b6446d098540ce590417c23/users/0bfe2bebfbd4476fabb2bac97e8aa9d4/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:14:30.205266 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad4913-4fd3-42e4-ba5f-97e4a7d833e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:30.205948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad4913-4fd3-42e4-ba5f-97e4a7d833e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:30.206287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad4913-4fd3-42e4-ba5f-97e4a7d833e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:30.206627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad4913-4fd3-42e4-ba5f-97e4a7d833e4 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23/users/0bfe2bebfbd4476fabb2bac97e8aa9d4/roles/31058b9633b44d7ca64f103119987fd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:30.208913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad4913-4fd3-42e4-ba5f-97e4a7d833e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=83919a658b6446d098540ce590417c23, user_id=0bfe2bebfbd4476fabb2bac97e8aa9d4, role_id=31058b9633b44d7ca64f103119987fd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:30.210440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad4913-4fd3-42e4-ba5f-97e4a7d833e4 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:30.232262 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 111/234] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:14:29 2022] PUT /identity/v3/projects/83919a658b6446d098540ce590417c23/users/0bfe2bebfbd4476fabb2bac97e8aa9d4/roles/31058b9633b44d7ca64f103119987fd9 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:14:30.890539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bf04e-c1e7-429f-bf2e-a88ed942c48c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:30.891076 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bf04e-c1e7-429f-bf2e-a88ed942c48c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:30.891076 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bf04e-c1e7-429f-bf2e-a88ed942c48c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:30.892747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 124/235] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:30.898136 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501d6aa-8f19-4918-a488-bb5218174ce5 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:30.898534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501d6aa-8f19-4918-a488-bb5218174ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:30.898718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501d6aa-8f19-4918-a488-bb5218174ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:30.917794 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-1501d6aa-8f19-4918-a488-bb5218174ce5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:30.919003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1501d6aa-8f19-4918-a488-bb5218174ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:31.011513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 112/236] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:31.077461 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb168c2-cd09-4407-830b-6742ba120bad None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:31.077760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb168c2-cd09-4407-830b-6742ba120bad None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:31.078015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb168c2-cd09-4407-830b-6742ba120bad None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:31.079401 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 125/237] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:31.084978 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6e31e-db5d-41d0-862c-50986c30b90b None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:31.085311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6e31e-db5d-41d0-862c-50986c30b90b None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:31.085613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6e31e-db5d-41d0-862c-50986c30b90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:31.104942 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-47b6e31e-db5d-41d0-862c-50986c30b90b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:31.106128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-47b6e31e-db5d-41d0-862c-50986c30b90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:31.173983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 113/238] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:31.426844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb168c2-cd09-4407-830b-6742ba120bad None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:31.461484 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6e31e-db5d-41d0-862c-50986c30b90b None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:31.480447 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:31.594224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb168c2-cd09-4407-830b-6742ba120bad None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:31.632366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6e31e-db5d-41d0-862c-50986c30b90b None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:31.660809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd34aba-ca38-4618-9268-6d2c43821125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:31.661456 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd34aba-ca38-4618-9268-6d2c43821125 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:31.661735 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd34aba-ca38-4618-9268-6d2c43821125 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:31.662004 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd34aba-ca38-4618-9268-6d2c43821125 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:31.673110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87a12a-fcb8-4a0f-b088-a32672a1a5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:31.673750 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87a12a-fcb8-4a0f-b088-a32672a1a5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:31.674033 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87a12a-fcb8-4a0f-b088-a32672a1a5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:31.674361 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87a12a-fcb8-4a0f-b088-a32672a1a5d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:31.676306 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87a12a-fcb8-4a0f-b088-a32672a1a5d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:31.678065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87a12a-fcb8-4a0f-b088-a32672a1a5d3 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:31.682716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 114/239] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:14:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:31.688565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd34aba-ca38-4618-9268-6d2c43821125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:31.692482 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87a12a-fcb8-4a0f-b088-a32672a1a5d3 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-9dd34aba-ca38-4618-9268-6d2c43821125 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:31.693673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:31.694549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:31.695331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:31.695331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:31.695331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:31.695331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:31.695331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:31.695331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:31.695331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:31.695331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:31.695331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:31.695331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:31.695331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:31.695331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:31.695331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 126/240] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:31.700727 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd34aba-ca38-4618-9268-6d2c43821125 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:31.711884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:31.809825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd34aba-ca38-4618-9268-6d2c43821125 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:31.841209 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:31.858617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87a12a-fcb8-4a0f-b088-a32672a1a5d3 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:31.874266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706322e1-ba28-4854-8569-aa7189c433a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:31.874886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706322e1-ba28-4854-8569-aa7189c433a7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:31.875172 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706322e1-ba28-4854-8569-aa7189c433a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:31.875454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706322e1-ba28-4854-8569-aa7189c433a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:31.876159 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706322e1-ba28-4854-8569-aa7189c433a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:31.876470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706322e1-ba28-4854-8569-aa7189c433a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:31.877988 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706322e1-ba28-4854-8569-aa7189c433a7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:31.886712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 127/241] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:31.892371 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f546-0adb-47da-82b8-c4b76fbb1770 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:31.892899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f546-0adb-47da-82b8-c4b76fbb1770 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:31.892899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f546-0adb-47da-82b8-c4b76fbb1770 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:31.894161 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 128/242] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:31.895349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516f546-0adb-47da-82b8-c4b76fbb1770 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:31.895762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927061d-8a11-43dd-8ef2-1372c0775d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:31.896650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927061d-8a11-43dd-8ef2-1372c0775d93 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:31.897112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927061d-8a11-43dd-8ef2-1372c0775d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:31.897589 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927061d-8a11-43dd-8ef2-1372c0775d93 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:31.898419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927061d-8a11-43dd-8ef2-1372c0775d93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:31.901004 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927061d-8a11-43dd-8ef2-1372c0775d93 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:31.927777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 115/243] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:14:31 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:14:31.929379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e4a78-2f5b-41d7-8ae9-6fbee3a97b60 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:31.929850 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e4a78-2f5b-41d7-8ae9-6fbee3a97b60 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:31.930348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e4a78-2f5b-41d7-8ae9-6fbee3a97b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:31.951305 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-a89e4a78-2f5b-41d7-8ae9-6fbee3a97b60 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:31.952930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a89e4a78-2f5b-41d7-8ae9-6fbee3a97b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:32.014148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:32.065006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 116/244] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:32.065431 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516f546-0adb-47da-82b8-c4b76fbb1770 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:32.103165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7251dd-2230-4824-a379-661c3c19bd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:32.104834 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7251dd-2230-4824-a379-661c3c19bd75 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:32.105299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7251dd-2230-4824-a379-661c3c19bd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:32.105831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7251dd-2230-4824-a379-661c3c19bd75 None admin] PATH_INFO: `/v3/users/6bcb57a7cde64b1b80fdf2b96eefce53` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:32.121872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7251dd-2230-4824-a379-661c3c19bd75 None admin] RBAC: Authorizing `identity:get_user(user_id=6bcb57a7cde64b1b80fdf2b96eefce53)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:32.124214 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7251dd-2230-4824-a379-661c3c19bd75 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:32.127708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 129/245] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:31 2022] GET /identity/v3/users/6bcb57a7cde64b1b80fdf2b96eefce53 => generated 297 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:32.133645 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e4a78-2f5b-41d7-8ae9-6fbee3a97b60 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:32.216080 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:32.249293 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e4a78-2f5b-41d7-8ae9-6fbee3a97b60 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:32.285908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cb836-bb01-4553-b509-8e1958222305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:32.286568 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cb836-bb01-4553-b509-8e1958222305 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:32.286862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cb836-bb01-4553-b509-8e1958222305 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:32.287192 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cb836-bb01-4553-b509-8e1958222305 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:32.372437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1cb836-bb01-4553-b509-8e1958222305 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:32.374026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1cb836-bb01-4553-b509-8e1958222305 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:32.450749 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:32.467708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7251dd-2230-4824-a379-661c3c19bd75 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:32.495592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 117/246] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:14:32.498418 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cb836-bb01-4553-b509-8e1958222305 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:32.596898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7251dd-2230-4824-a379-661c3c19bd75 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:32.632522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:32.645790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cb836-bb01-4553-b509-8e1958222305 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:32.651463 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d8ee0-6513-466f-9832-b194e30bb8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:32.652410 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d8ee0-6513-466f-9832-b194e30bb8ef None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:32.652889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d8ee0-6513-466f-9832-b194e30bb8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:32.653395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d8ee0-6513-466f-9832-b194e30bb8ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:32.669523 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d8ee0-6513-466f-9832-b194e30bb8ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6c0d8ee0-6513-466f-9832-b194e30bb8ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:32.677615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:32.678572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:32.679364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:32.679364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:32.679364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:32.679364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:32.679364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:32.679364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:32.679364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:32.679364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:32.679364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:32.679364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:32.679364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:32.679364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:32.681193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 130/247] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:32.688675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d8ee0-6513-466f-9832-b194e30bb8ef None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:32.696444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d32ce-7834-43bf-8ea8-f2d5fcaf9fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:32.697192 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d32ce-7834-43bf-8ea8-f2d5fcaf9fde None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:32.697492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d32ce-7834-43bf-8ea8-f2d5fcaf9fde None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:32.697753 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d32ce-7834-43bf-8ea8-f2d5fcaf9fde None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:32.699009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d32ce-7834-43bf-8ea8-f2d5fcaf9fde None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:32.700715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d32ce-7834-43bf-8ea8-f2d5fcaf9fde None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:32.705996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 118/248] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:32 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:32.709738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d32ce-7834-43bf-8ea8-f2d5fcaf9fde None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:32.769931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:32.837439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d8ee0-6513-466f-9832-b194e30bb8ef None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:32.875560 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a0969-8e51-4ff4-bf89-66e5aee2f5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:32.876544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a0969-8e51-4ff4-bf89-66e5aee2f5dc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:32.877086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a0969-8e51-4ff4-bf89-66e5aee2f5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:32.877496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a0969-8e51-4ff4-bf89-66e5aee2f5dc None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:32.878308 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a0969-8e51-4ff4-bf89-66e5aee2f5dc None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:32.878762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a0969-8e51-4ff4-bf89-66e5aee2f5dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:32.880301 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a0969-8e51-4ff4-bf89-66e5aee2f5dc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:32.882376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d32ce-7834-43bf-8ea8-f2d5fcaf9fde None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:32.890580 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 131/249] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:32.892166 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c7bd0-08a8-4ef9-ba1b-ebb77d1711a5 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:32.892590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c7bd0-08a8-4ef9-ba1b-ebb77d1711a5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:32.893435 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c7bd0-08a8-4ef9-ba1b-ebb77d1711a5 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:32.894926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 132/250] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:32.895781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c7bd0-08a8-4ef9-ba1b-ebb77d1711a5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:32.950885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:32.963263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd89ae-1db4-41c1-b8b6-2b51bf83c1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:32.964057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd89ae-1db4-41c1-b8b6-2b51bf83c1ad None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:32.964445 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd89ae-1db4-41c1-b8b6-2b51bf83c1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:32.964791 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd89ae-1db4-41c1-b8b6-2b51bf83c1ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:32.965598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd89ae-1db4-41c1-b8b6-2b51bf83c1ad None admin] RBAC: Adding query filter params (role.id=c6bc2b1713e84fecb6708753f6ba6a66, scope.project.id=83919a658b6446d098540ce590417c23, user.id=6bcb57a7cde64b1b80fdf2b96eefce53) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:32.965992 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd89ae-1db4-41c1-b8b6-2b51bf83c1ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:32.967555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd89ae-1db4-41c1-b8b6-2b51bf83c1ad None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:33.005656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 119/251] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:32 2022] GET /identity/v3/role_assignments?user.id=6bcb57a7cde64b1b80fdf2b96eefce53&scope.project.id=83919a658b6446d098540ce590417c23&role.id=c6bc2b1713e84fecb6708753f6ba6a66 => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:33.006940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4565774-a875-47f2-b923-c7b6a980e64c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:33.007226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4565774-a875-47f2-b923-c7b6a980e64c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:33.007505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4565774-a875-47f2-b923-c7b6a980e64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:33.008177 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c7bd0-08a8-4ef9-ba1b-ebb77d1711a5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:33.049928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b4565774-a875-47f2-b923-c7b6a980e64c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:33.052195 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b4565774-a875-47f2-b923-c7b6a980e64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:33.052986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca033837-60c7-41fe-9f20-9459846f5e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:33.053814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca033837-60c7-41fe-9f20-9459846f5e7f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:33.054261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca033837-60c7-41fe-9f20-9459846f5e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:33.054723 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca033837-60c7-41fe-9f20-9459846f5e7f None admin] PATH_INFO: `/v3/users/9500c1cf0d9f4b4c96ad281fbe5245e5` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:33.058338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca033837-60c7-41fe-9f20-9459846f5e7f None admin] RBAC: Authorizing `identity:get_user(user_id=9500c1cf0d9f4b4c96ad281fbe5245e5)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:33.060113 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca033837-60c7-41fe-9f20-9459846f5e7f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:33.062046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 133/252] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:32 2022] GET /identity/v3/users/9500c1cf0d9f4b4c96ad281fbe5245e5 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:33.068578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca033837-60c7-41fe-9f20-9459846f5e7f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:33.186682 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 120/253] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:33.193923 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca033837-60c7-41fe-9f20-9459846f5e7f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:33.214240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:33.234279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568efa1-14bb-4625-9250-d176eb3d8b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:33.235206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568efa1-14bb-4625-9250-d176eb3d8b88 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:33.235728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568efa1-14bb-4625-9250-d176eb3d8b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:33.236171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568efa1-14bb-4625-9250-d176eb3d8b88 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:33.237469 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568efa1-14bb-4625-9250-d176eb3d8b88 None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:33.239083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568efa1-14bb-4625-9250-d176eb3d8b88 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:33.241001 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 134/254] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:33 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:33.247429 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4565774-a875-47f2-b923-c7b6a980e64c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:33.304052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:33.396143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e996bc9-92ba-456a-8d0a-0a10215f0fec None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:33.396820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e996bc9-92ba-456a-8d0a-0a10215f0fec None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:33.397287 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e996bc9-92ba-456a-8d0a-0a10215f0fec None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:33.397614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4565774-a875-47f2-b923-c7b6a980e64c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:33.399318 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 135/255] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:33.404535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c4b7-3dad-4d3a-a841-d084ad84eb48 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:33.405056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c4b7-3dad-4d3a-a841-d084ad84eb48 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:33.405521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c4b7-3dad-4d3a-a841-d084ad84eb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:33.435595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f5fb5-5b24-4f4d-8264-e57c71624a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:33.435595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f5fb5-5b24-4f4d-8264-e57c71624a9e None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:33.435595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f5fb5-5b24-4f4d-8264-e57c71624a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:33.435941 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f5fb5-5b24-4f4d-8264-e57c71624a9e None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23/users/9500c1cf0d9f4b4c96ad281fbe5245e5/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:33.437826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f5fb5-5b24-4f4d-8264-e57c71624a9e None admin] RBAC: Authorizing `identity:create_grant(project_id=83919a658b6446d098540ce590417c23, user_id=9500c1cf0d9f4b4c96ad281fbe5245e5, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:33.439285 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f5fb5-5b24-4f4d-8264-e57c71624a9e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:33.447892 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5572c4b7-3dad-4d3a-a841-d084ad84eb48 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:33.450120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5572c4b7-3dad-4d3a-a841-d084ad84eb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:33.458095 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 121/256] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:14:33 2022] PUT /identity/v3/projects/83919a658b6446d098540ce590417c23/users/9500c1cf0d9f4b4c96ad281fbe5245e5/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:14:33.546623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 136/257] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:33.588086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f5fb5-5b24-4f4d-8264-e57c71624a9e None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:33.694198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:33.702522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f5fb5-5b24-4f4d-8264-e57c71624a9e None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:33.744598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eea9b8-0ff0-4240-be2e-ae4b307b765c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:33.745600 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eea9b8-0ff0-4240-be2e-ae4b307b765c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:33.746082 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eea9b8-0ff0-4240-be2e-ae4b307b765c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:33.746583 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eea9b8-0ff0-4240-be2e-ae4b307b765c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:33.758005 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eea9b8-0ff0-4240-be2e-ae4b307b765c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-76eea9b8-0ff0-4240-be2e-ae4b307b765c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:33.764049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:33.764928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:33.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:33.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:33.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:33.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:33.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:33.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:33.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:33.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:33.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:33.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:33.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:33.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:33.768334 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 122/258] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:14:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:33.776970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572c4b7-3dad-4d3a-a841-d084ad84eb48 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:33.889734 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572c4b7-3dad-4d3a-a841-d084ad84eb48 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:33.911477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:33.936890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ca1b1-63eb-4bf9-a187-1479e5623d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:33.937511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ca1b1-63eb-4bf9-a187-1479e5623d67 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:33.937798 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ca1b1-63eb-4bf9-a187-1479e5623d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:33.938072 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ca1b1-63eb-4bf9-a187-1479e5623d67 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:33.938752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ca1b1-63eb-4bf9-a187-1479e5623d67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:33.939057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ca1b1-63eb-4bf9-a187-1479e5623d67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:33.940455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ca1b1-63eb-4bf9-a187-1479e5623d67 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:33.954970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 137/259] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:33.964162 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eea9b8-0ff0-4240-be2e-ae4b307b765c None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:33.999675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ca1b1-63eb-4bf9-a187-1479e5623d67 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:34.129543 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eea9b8-0ff0-4240-be2e-ae4b307b765c None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:34.158491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ca1b1-63eb-4bf9-a187-1479e5623d67 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:34.195592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3284e70-b9d6-417b-a187-e38d06396a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:34.195592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3284e70-b9d6-417b-a187-e38d06396a6a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:34.195592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3284e70-b9d6-417b-a187-e38d06396a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:34.195592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3284e70-b9d6-417b-a187-e38d06396a6a None admin] PATH_INFO: `/v3/users/0bfe2bebfbd4476fabb2bac97e8aa9d4` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:34.196066 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3284e70-b9d6-417b-a187-e38d06396a6a None admin] RBAC: Authorizing `identity:get_user(user_id=0bfe2bebfbd4476fabb2bac97e8aa9d4)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:34.196777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3284e70-b9d6-417b-a187-e38d06396a6a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:34.199478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 123/260] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:33 2022] GET /identity/v3/users/0bfe2bebfbd4476fabb2bac97e8aa9d4 => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:34.200253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d5b17-0410-419d-bc31-0f0d127229d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:34.201114 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d5b17-0410-419d-bc31-0f0d127229d9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:34.201601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d5b17-0410-419d-bc31-0f0d127229d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:34.202061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d5b17-0410-419d-bc31-0f0d127229d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:34.207427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3284e70-b9d6-417b-a187-e38d06396a6a None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:34.229052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d5b17-0410-419d-bc31-0f0d127229d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8d4d5b17-0410-419d-bc31-0f0d127229d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:34.249031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:34.249976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:34.250764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:34.250764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:34.250764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:34.250764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:34.250764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:34.250764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:34.250764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:34.250764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:34.250764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:34.250764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:34.250764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:34.250764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:34.251365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:34.252854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 138/261] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:14:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:14:34.260293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d5b17-0410-419d-bc31-0f0d127229d9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:34.369515 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:34.405602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3284e70-b9d6-417b-a187-e38d06396a6a None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:34.497380 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abbfd0-1f3a-4eaf-911e-dec2884a0e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:34.498620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abbfd0-1f3a-4eaf-911e-dec2884a0e2e None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:34.499540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abbfd0-1f3a-4eaf-911e-dec2884a0e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:34.500031 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abbfd0-1f3a-4eaf-911e-dec2884a0e2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:34.504980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d5b17-0410-419d-bc31-0f0d127229d9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:34.629546 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420e1ec-6308-46e2-9489-73a942789cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:34.630586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420e1ec-6308-46e2-9489-73a942789cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:34.631082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420e1ec-6308-46e2-9489-73a942789cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:34.631587 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420e1ec-6308-46e2-9489-73a942789cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:34.632680 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abbfd0-1f3a-4eaf-911e-dec2884a0e2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:34.632812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420e1ec-6308-46e2-9489-73a942789cb9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:34.633174 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420e1ec-6308-46e2-9489-73a942789cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:34.635699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abbfd0-1f3a-4eaf-911e-dec2884a0e2e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:34.635939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420e1ec-6308-46e2-9489-73a942789cb9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:34.664049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 139/262] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:14:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:34.672994 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:34.673907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420e1ec-6308-46e2-9489-73a942789cb9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:34.799513 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:34.804527 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420e1ec-6308-46e2-9489-73a942789cb9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:34.840167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2755ae-5957-41dd-8cd9-823820a7bdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:34.841202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2755ae-5957-41dd-8cd9-823820a7bdfb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:34.841748 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2755ae-5957-41dd-8cd9-823820a7bdfb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:34.842214 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2755ae-5957-41dd-8cd9-823820a7bdfb None admin] PATH_INFO: `/v3/users/0bfe2bebfbd4476fabb2bac97e8aa9d4` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:34.843540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2755ae-5957-41dd-8cd9-823820a7bdfb None admin] RBAC: Authorizing `identity:get_user(user_id=0bfe2bebfbd4476fabb2bac97e8aa9d4)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:34.845183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2755ae-5957-41dd-8cd9-823820a7bdfb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:34.847249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 140/263] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:34 2022] GET /identity/v3/users/0bfe2bebfbd4476fabb2bac97e8aa9d4 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:34.852829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2755ae-5957-41dd-8cd9-823820a7bdfb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:34.896319 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 124/264] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:14:34.905550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abbfd0-1f3a-4eaf-911e-dec2884a0e2e None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:34.960925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2755ae-5957-41dd-8cd9-823820a7bdfb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:34.997112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c90d2-48a5-4d47-b40c-10875be1c38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:34.998050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c90d2-48a5-4d47-b40c-10875be1c38a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:34.998538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c90d2-48a5-4d47-b40c-10875be1c38a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:34.998999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c90d2-48a5-4d47-b40c-10875be1c38a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:35.041252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:35.120929 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c90d2-48a5-4d47-b40c-10875be1c38a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:35.122924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c90d2-48a5-4d47-b40c-10875be1c38a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:35.213383 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abbfd0-1f3a-4eaf-911e-dec2884a0e2e None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:35.256656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:35.274265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15172892-0815-4287-8e27-e345414edce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:35.275603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15172892-0815-4287-8e27-e345414edce0 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:35.276114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15172892-0815-4287-8e27-e345414edce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:35.276589 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15172892-0815-4287-8e27-e345414edce0 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:35.281197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15172892-0815-4287-8e27-e345414edce0 None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:35.283140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15172892-0815-4287-8e27-e345414edce0 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:35.285373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 125/265] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:34 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:35.291737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15172892-0815-4287-8e27-e345414edce0 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:35.357353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 141/266] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:14:35.371771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c90d2-48a5-4d47-b40c-10875be1c38a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:35.477634 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15172892-0815-4287-8e27-e345414edce0 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:35.482799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:35.536605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c90d2-48a5-4d47-b40c-10875be1c38a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:35.547527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c597486-88f1-458b-896b-68fef93cba6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:35.548264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c597486-88f1-458b-896b-68fef93cba6c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:35.548552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c597486-88f1-458b-896b-68fef93cba6c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:35.548828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c597486-88f1-458b-896b-68fef93cba6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:35.549583 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c597486-88f1-458b-896b-68fef93cba6c None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=83919a658b6446d098540ce590417c23, user.id=0bfe2bebfbd4476fabb2bac97e8aa9d4) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:35.549894 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c597486-88f1-458b-896b-68fef93cba6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:35.551387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c597486-88f1-458b-896b-68fef93cba6c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:35.564469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 126/267] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:35 2022] GET /identity/v3/role_assignments?user.id=0bfe2bebfbd4476fabb2bac97e8aa9d4&scope.project.id=83919a658b6446d098540ce590417c23&role.id=23538dc4c770454e80dd11ed8e498539 => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:35.595521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38f10d-80ae-4346-9e41-1fbf272987b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:35.596219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38f10d-80ae-4346-9e41-1fbf272987b3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:35.596499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38f10d-80ae-4346-9e41-1fbf272987b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:35.596773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38f10d-80ae-4346-9e41-1fbf272987b3 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:35.600425 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38f10d-80ae-4346-9e41-1fbf272987b3 None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:35.601933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38f10d-80ae-4346-9e41-1fbf272987b3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:35.603832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 142/268] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:35 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:35.609992 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c597486-88f1-458b-896b-68fef93cba6c None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:35.706452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:35.735483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd2f32-5b5e-49df-9a11-739067f6c33d None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:35.735814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd2f32-5b5e-49df-9a11-739067f6c33d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:35.736062 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd2f32-5b5e-49df-9a11-739067f6c33d None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:35.737404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 143/269] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:14:35.746765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1d7eb-0cd9-459d-bab1-25afae7125e4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:35.747077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1d7eb-0cd9-459d-bab1-25afae7125e4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:35.747359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1d7eb-0cd9-459d-bab1-25afae7125e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:35.760718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c597486-88f1-458b-896b-68fef93cba6c None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:35.791733 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0af1d7eb-0cd9-459d-bab1-25afae7125e4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:35.793156 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0af1d7eb-0cd9-459d-bab1-25afae7125e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:35.801647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf8348-9d5f-4d7b-ac29-2f269ef57c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:35.802272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf8348-9d5f-4d7b-ac29-2f269ef57c46 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:35.802548 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf8348-9d5f-4d7b-ac29-2f269ef57c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:35.802816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf8348-9d5f-4d7b-ac29-2f269ef57c46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:35.803554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf8348-9d5f-4d7b-ac29-2f269ef57c46 None admin] RBAC: Adding query filter params (role.id=31058b9633b44d7ca64f103119987fd9, scope.project.id=83919a658b6446d098540ce590417c23, user.id=0bfe2bebfbd4476fabb2bac97e8aa9d4) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:35.803956 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf8348-9d5f-4d7b-ac29-2f269ef57c46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:35.805439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf8348-9d5f-4d7b-ac29-2f269ef57c46 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:35.820175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 127/270] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:35 2022] GET /identity/v3/role_assignments?user.id=0bfe2bebfbd4476fabb2bac97e8aa9d4&scope.project.id=83919a658b6446d098540ce590417c23&role.id=31058b9633b44d7ca64f103119987fd9 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:35.915099 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 144/271] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:36.288518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf8348-9d5f-4d7b-ac29-2f269ef57c46 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:36.357312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0b9c8-42f4-456e-b6cd-be37e030fafa None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:36.358269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0b9c8-42f4-456e-b6cd-be37e030fafa None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:36.358778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0b9c8-42f4-456e-b6cd-be37e030fafa None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:36.360848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 145/272] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:36.371552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf31f15-7b48-4c56-87d6-4beba7139d0c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:36.372182 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf31f15-7b48-4c56-87d6-4beba7139d0c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:36.372647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf31f15-7b48-4c56-87d6-4beba7139d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:36.399090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7cf31f15-7b48-4c56-87d6-4beba7139d0c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:36.401346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7cf31f15-7b48-4c56-87d6-4beba7139d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:36.425164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf8348-9d5f-4d7b-ac29-2f269ef57c46 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:36.448803 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:36.472249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a777aa-2e87-45eb-b3a9-f178c4d1d038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:36.472904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a777aa-2e87-45eb-b3a9-f178c4d1d038 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:36.473181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a777aa-2e87-45eb-b3a9-f178c4d1d038 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:36.473462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a777aa-2e87-45eb-b3a9-f178c4d1d038 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:36.484149 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:36.485963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a777aa-2e87-45eb-b3a9-f178c4d1d038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-69a777aa-2e87-45eb-b3a9-f178c4d1d038 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:36.491862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:36.492790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:36.493579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:36.493579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:36.493579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:36.493579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:36.493579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:36.493579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:36.493579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:36.493579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:36.493579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:36.493579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:36.493579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:36.493579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:36.493579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 128/273] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:36.496325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 146/274] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:36.504827 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a777aa-2e87-45eb-b3a9-f178c4d1d038 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:36.615701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a777aa-2e87-45eb-b3a9-f178c4d1d038 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:36.650539 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a2c51-29d0-4463-8593-6a9f10a34d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:36.651357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a2c51-29d0-4463-8593-6a9f10a34d62 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:36.651825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a2c51-29d0-4463-8593-6a9f10a34d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:36.652233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a2c51-29d0-4463-8593-6a9f10a34d62 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:36.653052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a2c51-29d0-4463-8593-6a9f10a34d62 None admin] RBAC: Adding query filter params (name=member) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:36.653508 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a2c51-29d0-4463-8593-6a9f10a34d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:36.655141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a2c51-29d0-4463-8593-6a9f10a34d62 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:36.665530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 129/275] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:36.672839 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf31f15-7b48-4c56-87d6-4beba7139d0c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:36.785713 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041d43-ff62-4719-a4be-7e20381691b2 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:36.786058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041d43-ff62-4719-a4be-7e20381691b2 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:36.786425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041d43-ff62-4719-a4be-7e20381691b2 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:36.789554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 130/276] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:14:36.794314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041d43-ff62-4719-a4be-7e20381691b2 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:36.805243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf31f15-7b48-4c56-87d6-4beba7139d0c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:36.838145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:36.841135 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f2650-a31c-4c2b-9321-b8d9bc15b966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:36.841780 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f2650-a31c-4c2b-9321-b8d9bc15b966 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:36.842061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f2650-a31c-4c2b-9321-b8d9bc15b966 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:36.842335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f2650-a31c-4c2b-9321-b8d9bc15b966 None admin] PATH_INFO: `/v3/users/ff52417e1fb34652907ac31a537afd8d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:36.856539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f2650-a31c-4c2b-9321-b8d9bc15b966 None admin] RBAC: Authorizing `identity:get_user(user_id=ff52417e1fb34652907ac31a537afd8d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:36.858004 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f2650-a31c-4c2b-9321-b8d9bc15b966 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:36.859675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 147/277] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:36 2022] GET /identity/v3/users/ff52417e1fb34652907ac31a537afd8d => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:36.865309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0421b6-dc3d-4248-bf1d-11904a9bf2ae None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:36.865597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0421b6-dc3d-4248-bf1d-11904a9bf2ae None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:36.865926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0421b6-dc3d-4248-bf1d-11904a9bf2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:36.883788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6c0421b6-dc3d-4248-bf1d-11904a9bf2ae None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:36.885005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6c0421b6-dc3d-4248-bf1d-11904a9bf2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:36.904167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041d43-ff62-4719-a4be-7e20381691b2 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:36.934761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:36.942200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9763085-69e5-434c-b4e0-19d5fe36256c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:36.942837 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9763085-69e5-434c-b4e0-19d5fe36256c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:36.943121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9763085-69e5-434c-b4e0-19d5fe36256c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:36.943396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9763085-69e5-434c-b4e0-19d5fe36256c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:36.944529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9763085-69e5-434c-b4e0-19d5fe36256c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:36.945988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9763085-69e5-434c-b4e0-19d5fe36256c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:36.947652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 131/278] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:14:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:36.959280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 148/279] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:36.959280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257d3ca-f665-48c0-b5c0-6f55558e016c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:36.959280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257d3ca-f665-48c0-b5c0-6f55558e016c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:36.959280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257d3ca-f665-48c0-b5c0-6f55558e016c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:36.959280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 149/280] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:36.959280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257d3ca-f665-48c0-b5c0-6f55558e016c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:36.959741 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9763085-69e5-434c-b4e0-19d5fe36256c None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:37.062772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257d3ca-f665-48c0-b5c0-6f55558e016c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:37.075212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9763085-69e5-434c-b4e0-19d5fe36256c None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:37.084487 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:37.100630 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b19e5-730f-4555-b1be-321e5ac61e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:37.101237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b19e5-730f-4555-b1be-321e5ac61e90 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:37.101512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b19e5-730f-4555-b1be-321e5ac61e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:37.101976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b19e5-730f-4555-b1be-321e5ac61e90 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:37.102413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b19e5-730f-4555-b1be-321e5ac61e90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:37.103850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b19e5-730f-4555-b1be-321e5ac61e90 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:37.112446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f59ef-3a50-4f39-81bd-4c2466373284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:37.113048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f59ef-3a50-4f39-81bd-4c2466373284 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:37.113594 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f59ef-3a50-4f39-81bd-4c2466373284 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:37.113768 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f59ef-3a50-4f39-81bd-4c2466373284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:37.114587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f59ef-3a50-4f39-81bd-4c2466373284 None admin] RBAC: Adding query filter params (role.id=ebbe7434d1ec4bc4ba71737f55987dd9, user.id=ff52417e1fb34652907ac31a537afd8d, scope.system=all) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:37.114894 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f59ef-3a50-4f39-81bd-4c2466373284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:37.116486 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f59ef-3a50-4f39-81bd-4c2466373284 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:37.122562 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 132/281] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Sep 12 23:14:36 2022] GET /identity/v3/role_assignments?user.id=ff52417e1fb34652907ac31a537afd8d&scope.system=all&role.id=ebbe7434d1ec4bc4ba71737f55987dd9 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:37.127633 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 150/282] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:14:36 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:14:37.129026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e898623-ec71-42bb-bdfe-3f4ebaf6cda0 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:37.129321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e898623-ec71-42bb-bdfe-3f4ebaf6cda0 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:37.129601 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e898623-ec71-42bb-bdfe-3f4ebaf6cda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:37.158885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-1e898623-ec71-42bb-bdfe-3f4ebaf6cda0 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:37.160542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1e898623-ec71-42bb-bdfe-3f4ebaf6cda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:37.225293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b19e5-730f-4555-b1be-321e5ac61e90 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:37.253916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 133/283] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:37.285690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:37.352423 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b19e5-730f-4555-b1be-321e5ac61e90 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:37.391898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4518d6-bdab-473a-8d21-4c1638d4da21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:37.393286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4518d6-bdab-473a-8d21-4c1638d4da21 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:37.393764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4518d6-bdab-473a-8d21-4c1638d4da21 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:37.394232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4518d6-bdab-473a-8d21-4c1638d4da21 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:37.405575 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4518d6-bdab-473a-8d21-4c1638d4da21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-2b4518d6-bdab-473a-8d21-4c1638d4da21 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:37.411619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:37.412475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:37.413263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:37.413263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:37.413263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:37.413263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:37.413263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:37.413263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:37.413263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:37.413263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:37.413263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:37.413263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:37.413263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:37.413263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:37.416126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 151/284] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:37.421252 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e898623-ec71-42bb-bdfe-3f4ebaf6cda0 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:37.535884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4518d6-bdab-473a-8d21-4c1638d4da21 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:37.541940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e898623-ec71-42bb-bdfe-3f4ebaf6cda0 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:37.543670 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:37.586713 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5eca92-4dbb-4bd2-b6e5-2abc66374254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:37.587341 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5eca92-4dbb-4bd2-b6e5-2abc66374254 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:37.587721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5eca92-4dbb-4bd2-b6e5-2abc66374254 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:37.587997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5eca92-4dbb-4bd2-b6e5-2abc66374254 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:37.589390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5eca92-4dbb-4bd2-b6e5-2abc66374254 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:37.589701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5eca92-4dbb-4bd2-b6e5-2abc66374254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:37.591104 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5eca92-4dbb-4bd2-b6e5-2abc66374254 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:37.607139 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 134/285] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:37.610346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5eca92-4dbb-4bd2-b6e5-2abc66374254 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:37.664894 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:37.668516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4518d6-bdab-473a-8d21-4c1638d4da21 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:37.705782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b1aec-3129-4aeb-bab0-83a5ce1d73c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:37.706522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b1aec-3129-4aeb-bab0-83a5ce1d73c6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:37.706803 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b1aec-3129-4aeb-bab0-83a5ce1d73c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:37.707078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b1aec-3129-4aeb-bab0-83a5ce1d73c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:37.719444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b1aec-3129-4aeb-bab0-83a5ce1d73c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-7b4b1aec-3129-4aeb-bab0-83a5ce1d73c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:37.724693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:37.725596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:37.726387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:37.726387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:37.726387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:37.726387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:37.726387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:37.726387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:37.726387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:37.726387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:37.726387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:37.726387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:37.726387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:37.726387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:37.726387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 152/286] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:14:37.731930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b1aec-3129-4aeb-bab0-83a5ce1d73c6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:37.754598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5eca92-4dbb-4bd2-b6e5-2abc66374254 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:37.808917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1791093-6577-41d0-95e8-43e5404b398a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:37.809558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1791093-6577-41d0-95e8-43e5404b398a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:37.809838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1791093-6577-41d0-95e8-43e5404b398a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:37.810113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1791093-6577-41d0-95e8-43e5404b398a None admin] PATH_INFO: `/v3/users/6bcb57a7cde64b1b80fdf2b96eefce53` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:37.811241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1791093-6577-41d0-95e8-43e5404b398a None admin] RBAC: Authorizing `identity:get_user(user_id=6bcb57a7cde64b1b80fdf2b96eefce53)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:37.819902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1791093-6577-41d0-95e8-43e5404b398a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:37.821956 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 135/287] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:37 2022] GET /identity/v3/users/6bcb57a7cde64b1b80fdf2b96eefce53 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:37.836967 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1791093-6577-41d0-95e8-43e5404b398a None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:37.846483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b1aec-3129-4aeb-bab0-83a5ce1d73c6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:37.886273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f064253-5a0e-4a1b-8a92-a4564d6c811d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:37.886896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f064253-5a0e-4a1b-8a92-a4564d6c811d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:37.887175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f064253-5a0e-4a1b-8a92-a4564d6c811d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:37.887446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f064253-5a0e-4a1b-8a92-a4564d6c811d None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:37.888249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f064253-5a0e-4a1b-8a92-a4564d6c811d None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:37.888554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f064253-5a0e-4a1b-8a92-a4564d6c811d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:37.889945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f064253-5a0e-4a1b-8a92-a4564d6c811d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:37.902395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:37.905826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 153/288] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:37.915801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f064253-5a0e-4a1b-8a92-a4564d6c811d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:37.966675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:38.043804 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1791093-6577-41d0-95e8-43e5404b398a None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:38.091026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f064253-5a0e-4a1b-8a92-a4564d6c811d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:38.101445 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffcfad-9d53-4904-973b-0ce6672527c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:38.102101 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffcfad-9d53-4904-973b-0ce6672527c1 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:38.102378 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffcfad-9d53-4904-973b-0ce6672527c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:38.102653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffcfad-9d53-4904-973b-0ce6672527c1 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:38.108439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fffcfad-9d53-4904-973b-0ce6672527c1 None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:38.109960 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fffcfad-9d53-4904-973b-0ce6672527c1 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:38.111902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 136/289] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:37 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:38.119585 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffcfad-9d53-4904-973b-0ce6672527c1 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:38.131654 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f2b80-f296-4533-aa5b-1e9ddea55f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:38.132626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f2b80-f296-4533-aa5b-1e9ddea55f39 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:38.133122 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f2b80-f296-4533-aa5b-1e9ddea55f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:38.133581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f2b80-f296-4533-aa5b-1e9ddea55f39 None admin] PATH_INFO: `/v3/users/9500c1cf0d9f4b4c96ad281fbe5245e5` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:38.135262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f2b80-f296-4533-aa5b-1e9ddea55f39 None admin] RBAC: Authorizing `identity:get_user(user_id=9500c1cf0d9f4b4c96ad281fbe5245e5)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:38.137129 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f2b80-f296-4533-aa5b-1e9ddea55f39 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:38.139360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 154/290] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:37 2022] GET /identity/v3/users/9500c1cf0d9f4b4c96ad281fbe5245e5 => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:38.145308 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f2b80-f296-4533-aa5b-1e9ddea55f39 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:38.304383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f2b80-f296-4533-aa5b-1e9ddea55f39 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:38.305856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:38.311783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:38.313272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffcfad-9d53-4904-973b-0ce6672527c1 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:38.362284 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff415375-319e-4635-b141-7b7be340b908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:38.363810 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff415375-319e-4635-b141-7b7be340b908 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:38.364085 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff415375-319e-4635-b141-7b7be340b908 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:38.364382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff415375-319e-4635-b141-7b7be340b908 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:38.404371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa72d7-6a5e-45c3-b8f2-c5eec4c64fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:38.405048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa72d7-6a5e-45c3-b8f2-c5eec4c64fa6 None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:38.405330 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa72d7-6a5e-45c3-b8f2-c5eec4c64fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:38.405596 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa72d7-6a5e-45c3-b8f2-c5eec4c64fa6 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23/users/6bcb57a7cde64b1b80fdf2b96eefce53/roles/c6bc2b1713e84fecb6708753f6ba6a66` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:38.412903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa72d7-6a5e-45c3-b8f2-c5eec4c64fa6 None admin] RBAC: Authorizing `identity:create_grant(project_id=83919a658b6446d098540ce590417c23, user_id=6bcb57a7cde64b1b80fdf2b96eefce53, role_id=c6bc2b1713e84fecb6708753f6ba6a66)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:38.414380 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa72d7-6a5e-45c3-b8f2-c5eec4c64fa6 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:38.435935 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 137/291] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:14:38 2022] PUT /identity/v3/projects/83919a658b6446d098540ce590417c23/users/6bcb57a7cde64b1b80fdf2b96eefce53/roles/c6bc2b1713e84fecb6708753f6ba6a66 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:14:38.466048 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff415375-319e-4635-b141-7b7be340b908 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:38.467560 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff415375-319e-4635-b141-7b7be340b908 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:38.596132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:38.603959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 155/292] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:14:38.606523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa72d7-6a5e-45c3-b8f2-c5eec4c64fa6 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:38.747037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa72d7-6a5e-45c3-b8f2-c5eec4c64fa6 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:38.794772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf5466-5223-4719-ab85-558c748b442e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:38.795420 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf5466-5223-4719-ab85-558c748b442e None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:38.795726 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf5466-5223-4719-ab85-558c748b442e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:38.796002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf5466-5223-4719-ab85-558c748b442e None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:38.797205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf5466-5223-4719-ab85-558c748b442e None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:38.798680 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf5466-5223-4719-ab85-558c748b442e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:38.800512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 138/293] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:38 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:38.806350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff415375-319e-4635-b141-7b7be340b908 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:38.918213 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff415375-319e-4635-b141-7b7be340b908 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:38.980272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11364a-17e4-4b29-bcf3-77a26bf32f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:38.980879 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11364a-17e4-4b29-bcf3-77a26bf32f98 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:38.981158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11364a-17e4-4b29-bcf3-77a26bf32f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:38.981434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11364a-17e4-4b29-bcf3-77a26bf32f98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:38.982162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11364a-17e4-4b29-bcf3-77a26bf32f98 None admin] RBAC: Adding query filter params (role.id=ebbe7434d1ec4bc4ba71737f55987dd9, scope.project.id=83919a658b6446d098540ce590417c23, user.id=9500c1cf0d9f4b4c96ad281fbe5245e5) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:38.982472 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11364a-17e4-4b29-bcf3-77a26bf32f98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:38.983915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11364a-17e4-4b29-bcf3-77a26bf32f98 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:38.996216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 156/294] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:38 2022] GET /identity/v3/role_assignments?user.id=9500c1cf0d9f4b4c96ad281fbe5245e5&scope.project.id=83919a658b6446d098540ce590417c23&role.id=ebbe7434d1ec4bc4ba71737f55987dd9 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:39.746458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29d796-dbd7-4a84-95ec-cc7a5449641b None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:39.746985 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29d796-dbd7-4a84-95ec-cc7a5449641b None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:39.746985 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29d796-dbd7-4a84-95ec-cc7a5449641b None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:39.748637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 139/295] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:39.754033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f89656-1ce0-4495-a5dd-20f163bf0304 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:39.754431 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f89656-1ce0-4495-a5dd-20f163bf0304 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:39.754606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f89656-1ce0-4495-a5dd-20f163bf0304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:39.773714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f1f89656-1ce0-4495-a5dd-20f163bf0304 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:39.774994 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f1f89656-1ce0-4495-a5dd-20f163bf0304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:39.832052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:39.852305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 157/296] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:39.905024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab3521-200d-4540-9c18-037cd3b3f0a2 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:39.905674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab3521-200d-4540-9c18-037cd3b3f0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:39.906142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab3521-200d-4540-9c18-037cd3b3f0a2 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:39.908191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 140/297] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:39.920273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e659e-28cc-452c-9191-913ca720c2fe None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:39.920437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e659e-28cc-452c-9191-913ca720c2fe None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:39.920642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e659e-28cc-452c-9191-913ca720c2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:39.945908 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-676e659e-28cc-452c-9191-913ca720c2fe None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:39.947193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-676e659e-28cc-452c-9191-913ca720c2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:39.957615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0629915-a391-4947-a8a9-ae925f7817d9 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:39.957902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0629915-a391-4947-a8a9-ae925f7817d9 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:39.958179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0629915-a391-4947-a8a9-ae925f7817d9 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:39.959743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 141/298] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:39.965311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf852f1-dbaf-48a8-88f9-41848c599af4 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:39.965598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf852f1-dbaf-48a8-88f9-41848c599af4 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:39.965871 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf852f1-dbaf-48a8-88f9-41848c599af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:39.986582 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-aaf852f1-dbaf-48a8-88f9-41848c599af4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:39.987822 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-aaf852f1-dbaf-48a8-88f9-41848c599af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:40.018075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:40.020933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 158/299] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:40.069732 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 142/300] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:40.138402 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e659e-28cc-452c-9191-913ca720c2fe None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:40.245071 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e659e-28cc-452c-9191-913ca720c2fe None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:40.259675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:40.281149 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfda5c2-e1f3-4212-9a53-e4ff2e0fbe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:40.281710 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfda5c2-e1f3-4212-9a53-e4ff2e0fbe2d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:40.281987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfda5c2-e1f3-4212-9a53-e4ff2e0fbe2d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:40.282260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfda5c2-e1f3-4212-9a53-e4ff2e0fbe2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:40.293097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfda5c2-e1f3-4212-9a53-e4ff2e0fbe2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-3bfda5c2-e1f3-4212-9a53-e4ff2e0fbe2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:40.298323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:40.299198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:40.300033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:40.300033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:40.300033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:40.300033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:40.300033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:40.300033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:40.300033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:40.300033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:40.300033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:40.300033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:40.300033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:40.300033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:40.300033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 159/301] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:40.301636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf852f1-dbaf-48a8-88f9-41848c599af4 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:40.305393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfda5c2-e1f3-4212-9a53-e4ff2e0fbe2d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:40.414235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf852f1-dbaf-48a8-88f9-41848c599af4 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:40.422656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfda5c2-e1f3-4212-9a53-e4ff2e0fbe2d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:40.454963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdda865-dfa4-45cd-9508-fed67b4c5e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:40.455623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdda865-dfa4-45cd-9508-fed67b4c5e6f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:40.455894 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdda865-dfa4-45cd-9508-fed67b4c5e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:40.456161 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdda865-dfa4-45cd-9508-fed67b4c5e6f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:40.461288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33928c2c-5837-4cad-8031-517fea987a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:40.461895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33928c2c-5837-4cad-8031-517fea987a33 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:40.462179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33928c2c-5837-4cad-8031-517fea987a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:40.462474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33928c2c-5837-4cad-8031-517fea987a33 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:40.463751 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33928c2c-5837-4cad-8031-517fea987a33 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:40.464058 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33928c2c-5837-4cad-8031-517fea987a33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:40.465441 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33928c2c-5837-4cad-8031-517fea987a33 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:40.468166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdda865-dfa4-45cd-9508-fed67b4c5e6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-fbdda865-dfa4-45cd-9508-fed67b4c5e6f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:40.473267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:40.474125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:40.474899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:40.474899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:40.474899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:40.474899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:40.474899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:40.474899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:40.474899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:40.474899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:40.474899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:40.474899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:40.474899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:40.474899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:40.475493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 160/302] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:40.475493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33928c2c-5837-4cad-8031-517fea987a33 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:40.476765 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 143/303] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:14:40.482157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdda865-dfa4-45cd-9508-fed67b4c5e6f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:40.505672 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:40.521587 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:40.586597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdda865-dfa4-45cd-9508-fed67b4c5e6f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:40.595816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33928c2c-5837-4cad-8031-517fea987a33 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:40.619703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8e33d-b526-4ad6-9d9d-0962d9380ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:40.620337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e33d-b526-4ad6-9d9d-0962d9380ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:40.620620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e33d-b526-4ad6-9d9d-0962d9380ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:40.620892 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e33d-b526-4ad6-9d9d-0962d9380ae3 None admin] PATH_INFO: `/v3/users/ff52417e1fb34652907ac31a537afd8d` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:40.622027 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8e33d-b526-4ad6-9d9d-0962d9380ae3 None admin] RBAC: Authorizing `identity:get_user(user_id=ff52417e1fb34652907ac31a537afd8d)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:40.623475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8e33d-b526-4ad6-9d9d-0962d9380ae3 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:40.625236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 144/304] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:40 2022] GET /identity/v3/users/ff52417e1fb34652907ac31a537afd8d => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:40.625829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8e33d-b526-4ad6-9d9d-0962d9380ae3 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:40.635922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca0072-2c8e-4719-b1cc-415bee1bfe4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:40.636846 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca0072-2c8e-4719-b1cc-415bee1bfe4f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:40.637295 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca0072-2c8e-4719-b1cc-415bee1bfe4f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:40.637783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca0072-2c8e-4719-b1cc-415bee1bfe4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:40.639048 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca0072-2c8e-4719-b1cc-415bee1bfe4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:40.640736 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca0072-2c8e-4719-b1cc-415bee1bfe4f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:40.642379 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 161/305] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:14:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:40.647517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca0072-2c8e-4719-b1cc-415bee1bfe4f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:40.722550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:40.739632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8e33d-b526-4ad6-9d9d-0962d9380ae3 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:40.766815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca0072-2c8e-4719-b1cc-415bee1bfe4f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:40.770228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:40.774623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a708f-2b2a-4d3e-a5a5-169db629ae74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:40.775838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a708f-2b2a-4d3e-a5a5-169db629ae74 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:40.776185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a708f-2b2a-4d3e-a5a5-169db629ae74 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:40.776525 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a708f-2b2a-4d3e-a5a5-169db629ae74 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:40.777258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a708f-2b2a-4d3e-a5a5-169db629ae74 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:40.777563 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a708f-2b2a-4d3e-a5a5-169db629ae74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:40.778942 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a708f-2b2a-4d3e-a5a5-169db629ae74 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:40.792651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 145/306] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:40.792651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a708f-2b2a-4d3e-a5a5-169db629ae74 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:40.812887 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03291d38-7430-46a8-b714-fd83777bdf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:40.813545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03291d38-7430-46a8-b714-fd83777bdf53 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:40.813830 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03291d38-7430-46a8-b714-fd83777bdf53 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:40.814108 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03291d38-7430-46a8-b714-fd83777bdf53 None admin] PATH_INFO: `/v3/system/users/ff52417e1fb34652907ac31a537afd8d/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:40.815513 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03291d38-7430-46a8-b714-fd83777bdf53 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ff52417e1fb34652907ac31a537afd8d, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:40.816929 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03291d38-7430-46a8-b714-fd83777bdf53 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:40.825218 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 162/307] 10.222.0.27 () {58 vars in 1334 bytes} [Mon Sep 12 23:14:40 2022] PUT /identity/v3/system/users/ff52417e1fb34652907ac31a537afd8d/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:14:40.826037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03291d38-7430-46a8-b714-fd83777bdf53 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:40.899440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a708f-2b2a-4d3e-a5a5-169db629ae74 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:40.938721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec008860-7871-4257-ac88-79f42e7b3c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:40.939347 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec008860-7871-4257-ac88-79f42e7b3c90 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:40.939652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec008860-7871-4257-ac88-79f42e7b3c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:40.939919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec008860-7871-4257-ac88-79f42e7b3c90 None admin] PATH_INFO: `/v3/groups` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:40.940679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec008860-7871-4257-ac88-79f42e7b3c90 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:40.942153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec008860-7871-4257-ac88-79f42e7b3c90 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:40.953846 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03291d38-7430-46a8-b714-fd83777bdf53 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:40.967615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 146/308] 10.222.0.27 () {60 vars in 1134 bytes} [Mon Sep 12 23:14:40 2022] POST /identity/v3/groups => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:14:40.967615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd973b53-f6b6-4494-8087-c64158f6ee95 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:40.967615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd973b53-f6b6-4494-8087-c64158f6ee95 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:40.967615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd973b53-f6b6-4494-8087-c64158f6ee95 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:40.967615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 147/309] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:40.970733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5fff8-472d-474d-960e-3a4009c9415b None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:40.970733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5fff8-472d-474d-960e-3a4009c9415b None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:40.970733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5fff8-472d-474d-960e-3a4009c9415b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:40.991500 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-88a5fff8-472d-474d-960e-3a4009c9415b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:40.992923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-88a5fff8-472d-474d-960e-3a4009c9415b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:40.996354 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:41.008796 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5d07e-991a-4bb4-8105-7ccdb0b848f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:41.009670 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5d07e-991a-4bb4-8105-7ccdb0b848f7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:41.010318 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5d07e-991a-4bb4-8105-7ccdb0b848f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:41.010604 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5d07e-991a-4bb4-8105-7ccdb0b848f7 None admin] PATH_INFO: `/v3/users/fcfa86e091284f6e8c291ed507390f0b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:41.027330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5d07e-991a-4bb4-8105-7ccdb0b848f7 None admin] RBAC: Authorizing `identity:get_user(user_id=fcfa86e091284f6e8c291ed507390f0b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:41.028926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5d07e-991a-4bb4-8105-7ccdb0b848f7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:41.030979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 163/310] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:40 2022] GET /identity/v3/users/fcfa86e091284f6e8c291ed507390f0b => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:41.037646 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5d07e-991a-4bb4-8105-7ccdb0b848f7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:41.071355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:41.095766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 148/311] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:41.173818 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5d07e-991a-4bb4-8105-7ccdb0b848f7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:41.209173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64487fbf-2d6c-4370-b8f7-bc702730d826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:41.209792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64487fbf-2d6c-4370-b8f7-bc702730d826 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:41.210074 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64487fbf-2d6c-4370-b8f7-bc702730d826 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:41.210347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64487fbf-2d6c-4370-b8f7-bc702730d826 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:41.211077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64487fbf-2d6c-4370-b8f7-bc702730d826 None admin] RBAC: Adding query filter params (role.id=c6bc2b1713e84fecb6708753f6ba6a66, user.id=fcfa86e091284f6e8c291ed507390f0b, scope.system=all) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:41.211415 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64487fbf-2d6c-4370-b8f7-bc702730d826 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:41.212904 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64487fbf-2d6c-4370-b8f7-bc702730d826 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:41.220835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 164/312] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Sep 12 23:14:41 2022] GET /identity/v3/role_assignments?user.id=fcfa86e091284f6e8c291ed507390f0b&scope.system=all&role.id=c6bc2b1713e84fecb6708753f6ba6a66 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:41.372969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5fff8-472d-474d-960e-3a4009c9415b None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:41.476982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5fff8-472d-474d-960e-3a4009c9415b None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:41.512819 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cf0a5-9074-4a9e-bcbf-0ca24177ef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:41.513468 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cf0a5-9074-4a9e-bcbf-0ca24177ef99 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:41.513749 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cf0a5-9074-4a9e-bcbf-0ca24177ef99 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:41.514022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cf0a5-9074-4a9e-bcbf-0ca24177ef99 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:41.521091 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:41.524644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27cf0a5-9074-4a9e-bcbf-0ca24177ef99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-d27cf0a5-9074-4a9e-bcbf-0ca24177ef99 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:41.529877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:41.531121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:41.532255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:41.532255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:41.532255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:41.532255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:41.532255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:41.532255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:41.532255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:41.532255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:41.532255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:41.532255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:41.532255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:41.532255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:41.539944 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 149/313] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:41.542067 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64487fbf-2d6c-4370-b8f7-bc702730d826 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:41.637211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:41.647316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64487fbf-2d6c-4370-b8f7-bc702730d826 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:41.684150 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5b4b6-7040-484c-85a0-79deba515845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:41.684763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5b4b6-7040-484c-85a0-79deba515845 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:41.685044 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5b4b6-7040-484c-85a0-79deba515845 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:41.685318 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5b4b6-7040-484c-85a0-79deba515845 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:41.685983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5b4b6-7040-484c-85a0-79deba515845 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:41.686289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5b4b6-7040-484c-85a0-79deba515845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:41.687680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5b4b6-7040-484c-85a0-79deba515845 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:41.699675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 165/314] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:41.702089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cf0a5-9074-4a9e-bcbf-0ca24177ef99 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:41.807409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cf0a5-9074-4a9e-bcbf-0ca24177ef99 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:41.843257 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b47dca-abad-4293-8932-25dfdb7d9ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:41.843918 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b47dca-abad-4293-8932-25dfdb7d9ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:41.844205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b47dca-abad-4293-8932-25dfdb7d9ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:41.844483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b47dca-abad-4293-8932-25dfdb7d9ab5 None admin] PATH_INFO: `/v3/users/6bcb57a7cde64b1b80fdf2b96eefce53` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:41.845607 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b47dca-abad-4293-8932-25dfdb7d9ab5 None admin] RBAC: Authorizing `identity:get_user(user_id=6bcb57a7cde64b1b80fdf2b96eefce53)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:41.847053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b47dca-abad-4293-8932-25dfdb7d9ab5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:41.849021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 150/315] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:41 2022] GET /identity/v3/users/6bcb57a7cde64b1b80fdf2b96eefce53 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:41.854804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5b4b6-7040-484c-85a0-79deba515845 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:41.953246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5b4b6-7040-484c-85a0-79deba515845 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:41.990355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7e641-d1e8-4e5b-a7d8-55b6426c643e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:41.990990 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7e641-d1e8-4e5b-a7d8-55b6426c643e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:41.991269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7e641-d1e8-4e5b-a7d8-55b6426c643e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:41.991543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7e641-d1e8-4e5b-a7d8-55b6426c643e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:42.015760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:42.063563 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7e641-d1e8-4e5b-a7d8-55b6426c643e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:42.064960 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7e641-d1e8-4e5b-a7d8-55b6426c643e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:42.168145 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 166/316] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:42.173388 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b47dca-abad-4293-8932-25dfdb7d9ab5 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:42.243221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:42.278995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b47dca-abad-4293-8932-25dfdb7d9ab5 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:42.313880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bdb40-91a4-4e71-8e42-2016cc6f4155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:42.314433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bdb40-91a4-4e71-8e42-2016cc6f4155 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:42.314712 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bdb40-91a4-4e71-8e42-2016cc6f4155 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:42.314986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bdb40-91a4-4e71-8e42-2016cc6f4155 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:42.316129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bdb40-91a4-4e71-8e42-2016cc6f4155 None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:42.317623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bdb40-91a4-4e71-8e42-2016cc6f4155 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:42.319296 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 151/317] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:42 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:42.325958 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7e641-d1e8-4e5b-a7d8-55b6426c643e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:42.380251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:42.427658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7e641-d1e8-4e5b-a7d8-55b6426c643e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:42.463554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be85403-aa5c-4d68-8cda-ecd250c82010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:42.464232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be85403-aa5c-4d68-8cda-ecd250c82010 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:42.464517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be85403-aa5c-4d68-8cda-ecd250c82010 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:42.464789 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be85403-aa5c-4d68-8cda-ecd250c82010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:42.465522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be85403-aa5c-4d68-8cda-ecd250c82010 None admin] RBAC: Adding query filter params (role.id=c6bc2b1713e84fecb6708753f6ba6a66, scope.project.id=83919a658b6446d098540ce590417c23, user.id=6bcb57a7cde64b1b80fdf2b96eefce53) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:42.465832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be85403-aa5c-4d68-8cda-ecd250c82010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:42.467262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be85403-aa5c-4d68-8cda-ecd250c82010 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:42.479726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 167/318] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:14:42 2022] GET /identity/v3/role_assignments?user.id=6bcb57a7cde64b1b80fdf2b96eefce53&scope.project.id=83919a658b6446d098540ce590417c23&role.id=c6bc2b1713e84fecb6708753f6ba6a66 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:43.042409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762a0fb-85ba-4a5c-9242-63503cf9616c None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:43.042952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762a0fb-85ba-4a5c-9242-63503cf9616c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:43.042952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762a0fb-85ba-4a5c-9242-63503cf9616c None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:43.044483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 152/319] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:43.049559 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb4cf4-a35b-44c0-a6ab-5fed5d1128ab None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:43.049969 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb4cf4-a35b-44c0-a6ab-5fed5d1128ab None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:43.050133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb4cf4-a35b-44c0-a6ab-5fed5d1128ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:43.068196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-79cb4cf4-a35b-44c0-a6ab-5fed5d1128ab None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:43.069357 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-79cb4cf4-a35b-44c0-a6ab-5fed5d1128ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:43.138247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 168/320] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:43.327657 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e38f3-4e1d-4a17-93c4-86b892e19d22 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:43.327982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e38f3-4e1d-4a17-93c4-86b892e19d22 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:43.328154 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e38f3-4e1d-4a17-93c4-86b892e19d22 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:43.329489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 153/321] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:43.334661 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571f279-37bc-48dd-bf00-67c8703cb648 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:43.334951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571f279-37bc-48dd-bf00-67c8703cb648 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:43.335220 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571f279-37bc-48dd-bf00-67c8703cb648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:43.352761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3571f279-37bc-48dd-bf00-67c8703cb648 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:43.354461 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3571f279-37bc-48dd-bf00-67c8703cb648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:43.399715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:43.414045 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e38f3-4e1d-4a17-93c4-86b892e19d22 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:43.431761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 169/322] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:43.493357 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb0f33-6831-4905-9d1c-7b47df6deb25 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:43.493709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb0f33-6831-4905-9d1c-7b47df6deb25 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:43.494510 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb0f33-6831-4905-9d1c-7b47df6deb25 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:43.496011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 170/323] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:14:43.500631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e58dee-cbf5-4c0c-ad1e-9b8b6e55d8a5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:43.500918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e58dee-cbf5-4c0c-ad1e-9b8b6e55d8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:43.501196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e58dee-cbf5-4c0c-ad1e-9b8b6e55d8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:43.519408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b9e58dee-cbf5-4c0c-ad1e-9b8b6e55d8a5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:43.520653 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b9e58dee-cbf5-4c0c-ad1e-9b8b6e55d8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:43.529493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e38f3-4e1d-4a17-93c4-86b892e19d22 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:43.566181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea5b6-d785-40ac-b17b-ca5e485bce79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:43.566809 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea5b6-d785-40ac-b17b-ca5e485bce79 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:43.567116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea5b6-d785-40ac-b17b-ca5e485bce79 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:43.567396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea5b6-d785-40ac-b17b-ca5e485bce79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:43.569301 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:43.579630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ea5b6-d785-40ac-b17b-ca5e485bce79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-6c0ea5b6-d785-40ac-b17b-ca5e485bce79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:43.584833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:43.585679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:43.586463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:43.586463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:43.586463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:43.586463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:43.586463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:43.586463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:43.586463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:43.586463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:43.586463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:43.586463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:43.586463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:43.586463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:43.586463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 154/324] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:43.591886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea5b6-d785-40ac-b17b-ca5e485bce79 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:43.592285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 171/325] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:43.689831 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea5b6-d785-40ac-b17b-ca5e485bce79 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:43.709692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e58dee-cbf5-4c0c-ad1e-9b8b6e55d8a5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:43.729354 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783fa41-a82d-4ea8-a4ac-47163ba4ffdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:43.729963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783fa41-a82d-4ea8-a4ac-47163ba4ffdb None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:43.730247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783fa41-a82d-4ea8-a4ac-47163ba4ffdb None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:43.730520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783fa41-a82d-4ea8-a4ac-47163ba4ffdb None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:43.731185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f783fa41-a82d-4ea8-a4ac-47163ba4ffdb None admin] RBAC: Adding query filter params (name=member) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:43.731480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f783fa41-a82d-4ea8-a4ac-47163ba4ffdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:43.732913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f783fa41-a82d-4ea8-a4ac-47163ba4ffdb None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:43.741466 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 155/326] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:43.746907 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783fa41-a82d-4ea8-a4ac-47163ba4ffdb None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:43.810421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:43.823644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e58dee-cbf5-4c0c-ad1e-9b8b6e55d8a5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:43.832944 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:43.857133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783fa41-a82d-4ea8-a4ac-47163ba4ffdb None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:43.868821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9f984-c041-426e-8538-0d6f6dad08fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:43.869453 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9f984-c041-426e-8538-0d6f6dad08fc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:43.869734 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9f984-c041-426e-8538-0d6f6dad08fc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:43.870010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9f984-c041-426e-8538-0d6f6dad08fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:43.871115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9f984-c041-426e-8538-0d6f6dad08fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:43.872574 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9f984-c041-426e-8538-0d6f6dad08fc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:43.875601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 172/327] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:14:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:43.875601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9f984-c041-426e-8538-0d6f6dad08fc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:43.892150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae03ff-991e-4afa-ae7f-05deab65ba90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:43.892784 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae03ff-991e-4afa-ae7f-05deab65ba90 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:43.893062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae03ff-991e-4afa-ae7f-05deab65ba90 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:43.893344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae03ff-991e-4afa-ae7f-05deab65ba90 None admin] PATH_INFO: `/v3/users/ff52417e1fb34652907ac31a537afd8d` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:43.894460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae03ff-991e-4afa-ae7f-05deab65ba90 None admin] RBAC: Authorizing `identity:get_user(user_id=ff52417e1fb34652907ac31a537afd8d)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:43.895935 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae03ff-991e-4afa-ae7f-05deab65ba90 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:43.902795 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 156/328] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:43 2022] GET /identity/v3/users/ff52417e1fb34652907ac31a537afd8d => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:43.902795 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae03ff-991e-4afa-ae7f-05deab65ba90 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:43.975481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9f984-c041-426e-8538-0d6f6dad08fc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:44.002511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae03ff-991e-4afa-ae7f-05deab65ba90 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:44.009040 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b708e0-e373-497a-aad8-fd5b182a9f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:44.009696 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b708e0-e373-497a-aad8-fd5b182a9f58 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:44.009974 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b708e0-e373-497a-aad8-fd5b182a9f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:44.010250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b708e0-e373-497a-aad8-fd5b182a9f58 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:44.020237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b708e0-e373-497a-aad8-fd5b182a9f58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:44.024725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c3b708e0-e373-497a-aad8-fd5b182a9f58 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:44.025659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:44.026597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:44.027380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:44.027380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:44.027380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:44.027380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:44.027380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:44.027380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:44.027380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:44.027380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:44.027380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:44.027380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:44.027380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:44.027380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:44.027380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 173/329] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:44.028174 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b708e0-e373-497a-aad8-fd5b182a9f58 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:44.042024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c90bc-0091-4a5e-8597-b67ffa0d0d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:44.042640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c90bc-0091-4a5e-8597-b67ffa0d0d11 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:44.043164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c90bc-0091-4a5e-8597-b67ffa0d0d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:44.043164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c90bc-0091-4a5e-8597-b67ffa0d0d11 None admin] PATH_INFO: `/v3/groups` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:44.043854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c90bc-0091-4a5e-8597-b67ffa0d0d11 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:44.045336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c90bc-0091-4a5e-8597-b67ffa0d0d11 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:44.059780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 157/330] 10.222.0.27 () {60 vars in 1134 bytes} [Mon Sep 12 23:14:43 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:14:44.060654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c90bc-0091-4a5e-8597-b67ffa0d0d11 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:44.084352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:44.142571 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b708e0-e373-497a-aad8-fd5b182a9f58 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:44.183401 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9416b-ce18-48db-b69b-6ed98c56cf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:44.184501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9416b-ce18-48db-b69b-6ed98c56cf20 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:44.184773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9416b-ce18-48db-b69b-6ed98c56cf20 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:44.184985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9416b-ce18-48db-b69b-6ed98c56cf20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:44.185739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9416b-ce18-48db-b69b-6ed98c56cf20 None admin] RBAC: Adding query filter params (role.id=ebbe7434d1ec4bc4ba71737f55987dd9, user.id=ff52417e1fb34652907ac31a537afd8d, scope.system=all) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:44.186056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9416b-ce18-48db-b69b-6ed98c56cf20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:44.186648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c90bc-0091-4a5e-8597-b67ffa0d0d11 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:44.187532 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9416b-ce18-48db-b69b-6ed98c56cf20 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:44.200693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 174/331] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Sep 12 23:14:44 2022] GET /identity/v3/role_assignments?user.id=ff52417e1fb34652907ac31a537afd8d&scope.system=all&role.id=ebbe7434d1ec4bc4ba71737f55987dd9 => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:44.225743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9adf6df-a426-46f3-8218-44a7b5a7c57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:44.226404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adf6df-a426-46f3-8218-44a7b5a7c57e None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:44.226688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adf6df-a426-46f3-8218-44a7b5a7c57e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:44.226961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adf6df-a426-46f3-8218-44a7b5a7c57e None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:44.227685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9adf6df-a426-46f3-8218-44a7b5a7c57e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:44.228519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9adf6df-a426-46f3-8218-44a7b5a7c57e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:44.229969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9adf6df-a426-46f3-8218-44a7b5a7c57e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:44.241516 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 158/332] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:44.247165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9416b-ce18-48db-b69b-6ed98c56cf20 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:44.356728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:44.367847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9416b-ce18-48db-b69b-6ed98c56cf20 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:44.403439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b94119-dcdd-4ce4-825b-f296b5ad86f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:44.404101 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b94119-dcdd-4ce4-825b-f296b5ad86f3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:44.404380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b94119-dcdd-4ce4-825b-f296b5ad86f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:44.404656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b94119-dcdd-4ce4-825b-f296b5ad86f3 None admin] PATH_INFO: `/v3/users/fcfa86e091284f6e8c291ed507390f0b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:44.405780 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b94119-dcdd-4ce4-825b-f296b5ad86f3 None admin] RBAC: Authorizing `identity:get_user(user_id=fcfa86e091284f6e8c291ed507390f0b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:44.407206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b94119-dcdd-4ce4-825b-f296b5ad86f3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:44.409310 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 175/333] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:44 2022] GET /identity/v3/users/fcfa86e091284f6e8c291ed507390f0b => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:44.414756 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9adf6df-a426-46f3-8218-44a7b5a7c57e None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:44.464867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:44.519474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9adf6df-a426-46f3-8218-44a7b5a7c57e None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:44.557200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695a929-645f-46b3-a801-06c02fc10be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:44.558400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695a929-645f-46b3-a801-06c02fc10be4 None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:44.558893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695a929-645f-46b3-a801-06c02fc10be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:44.559379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695a929-645f-46b3-a801-06c02fc10be4 None admin] PATH_INFO: `/v3/system/users/fcfa86e091284f6e8c291ed507390f0b/roles/c6bc2b1713e84fecb6708753f6ba6a66` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:44.561318 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695a929-645f-46b3-a801-06c02fc10be4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=fcfa86e091284f6e8c291ed507390f0b, role_id=c6bc2b1713e84fecb6708753f6ba6a66)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:44.562904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695a929-645f-46b3-a801-06c02fc10be4 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:44.574738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 159/334] 10.222.0.27 () {58 vars in 1334 bytes} [Mon Sep 12 23:14:44 2022] PUT /identity/v3/system/users/fcfa86e091284f6e8c291ed507390f0b/roles/c6bc2b1713e84fecb6708753f6ba6a66 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:14:46.471128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347d4ac-5270-4af2-978f-67afc39073cf None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:46.471721 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347d4ac-5270-4af2-978f-67afc39073cf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:46.471721 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347d4ac-5270-4af2-978f-67afc39073cf None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:46.473171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 176/335] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:46.478861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dfa85-80b0-4926-a1cf-fd12cb2fb14e None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:46.479244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dfa85-80b0-4926-a1cf-fd12cb2fb14e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:46.479436 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dfa85-80b0-4926-a1cf-fd12cb2fb14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:46.508572 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-a70dfa85-80b0-4926-a1cf-fd12cb2fb14e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:46.510303 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a70dfa85-80b0-4926-a1cf-fd12cb2fb14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:46.602192 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 160/336] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:47.152100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347d4ac-5270-4af2-978f-67afc39073cf None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:47.274226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea37bd8-61dd-471d-94b5-451859a4f7e1 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:47.274565 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea37bd8-61dd-471d-94b5-451859a4f7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:47.274782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea37bd8-61dd-471d-94b5-451859a4f7e1 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:47.277263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 161/337] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:47.279922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347d4ac-5270-4af2-978f-67afc39073cf None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:47.284026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1130a4d-d54f-4824-9dc0-b8e2ee995bcd None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:47.284312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1130a4d-d54f-4824-9dc0-b8e2ee995bcd None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:47.284592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1130a4d-d54f-4824-9dc0-b8e2ee995bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:47.319516 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c1130a4d-d54f-4824-9dc0-b8e2ee995bcd None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:47.320871 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c1130a4d-d54f-4824-9dc0-b8e2ee995bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:47.350199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30631707-ec18-4397-9092-fe5d2aff5ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:47.350848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30631707-ec18-4397-9092-fe5d2aff5ddf None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:47.351128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30631707-ec18-4397-9092-fe5d2aff5ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:47.351400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30631707-ec18-4397-9092-fe5d2aff5ddf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:47.368284 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30631707-ec18-4397-9092-fe5d2aff5ddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-30631707-ec18-4397-9092-fe5d2aff5ddf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:47.375740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:47.376649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:47.377445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:47.377445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:47.377445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:47.377445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:47.377445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:47.377445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:47.377445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:47.377445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:47.377445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:47.377445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:47.377445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:47.377445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:47.377445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 177/338] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:47.381499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:47.390892 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb54c8-ac52-4d5c-9b64-67dbb0db314a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:47.391171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb54c8-ac52-4d5c-9b64-67dbb0db314a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:47.391444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb54c8-ac52-4d5c-9b64-67dbb0db314a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:47.393161 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 178/339] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:47 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:47.393906 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb54c8-ac52-4d5c-9b64-67dbb0db314a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:47.427275 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:47.441674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 162/340] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:47.443258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d90b7-b449-47b8-bfe8-2401a32b9e53 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:47.443545 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d90b7-b449-47b8-bfe8-2401a32b9e53 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:47.444657 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d90b7-b449-47b8-bfe8-2401a32b9e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:47.482210 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-332d90b7-b449-47b8-bfe8-2401a32b9e53 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:47.483532 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-332d90b7-b449-47b8-bfe8-2401a32b9e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:47.567457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 163/341] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:47.585189 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb54c8-ac52-4d5c-9b64-67dbb0db314a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:47.633451 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe607e83-cce0-4355-9ad4-642834edd7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:47.634488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe607e83-cce0-4355-9ad4-642834edd7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:47.634963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe607e83-cce0-4355-9ad4-642834edd7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:47.635446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe607e83-cce0-4355-9ad4-642834edd7e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:47.636320 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe607e83-cce0-4355-9ad4-642834edd7e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:47.636813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe607e83-cce0-4355-9ad4-642834edd7e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:47.638359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe607e83-cce0-4355-9ad4-642834edd7e6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:47.651709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 179/342] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:47.657476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d90b7-b449-47b8-bfe8-2401a32b9e53 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:47.799274 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:47.806047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d90b7-b449-47b8-bfe8-2401a32b9e53 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:47.852727 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf775416-a137-4f8a-b6e6-c6ba5f06cb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:47.853834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf775416-a137-4f8a-b6e6-c6ba5f06cb1b None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:47.854115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf775416-a137-4f8a-b6e6-c6ba5f06cb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:47.854389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf775416-a137-4f8a-b6e6-c6ba5f06cb1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:47.935294 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf775416-a137-4f8a-b6e6-c6ba5f06cb1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:47.936868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf775416-a137-4f8a-b6e6-c6ba5f06cb1b None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:47.981827 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe607e83-cce0-4355-9ad4-642834edd7e6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:48.079549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:48.090591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 164/343] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:48.090591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a6eb9-6421-442a-88b0-ee4cb4115202 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:48.090591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a6eb9-6421-442a-88b0-ee4cb4115202 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:48.090591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a6eb9-6421-442a-88b0-ee4cb4115202 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:48.093462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 165/344] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:48.093462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a6eb9-6421-442a-88b0-ee4cb4115202 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:48.130764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:48.146247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe607e83-cce0-4355-9ad4-642834edd7e6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:48.198921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ababfe6-8dd0-4165-aa95-240bb8525a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:48.199947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ababfe6-8dd0-4165-aa95-240bb8525a1c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:48.200401 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ababfe6-8dd0-4165-aa95-240bb8525a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:48.200852 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ababfe6-8dd0-4165-aa95-240bb8525a1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:48.211914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ababfe6-8dd0-4165-aa95-240bb8525a1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-7ababfe6-8dd0-4165-aa95-240bb8525a1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:48.217007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:48.218057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:48.218885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:48.218885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:48.218885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:48.218885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:48.218885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:48.218885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:48.218885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:48.218885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:48.218885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:48.218885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:48.218885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:48.218885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:48.222120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 180/345] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:14:48.222251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ababfe6-8dd0-4165-aa95-240bb8525a1c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:48.228574 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a6eb9-6421-442a-88b0-ee4cb4115202 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:48.272108 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62655bfd-a8c0-4f97-9e12-4b4f93996198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:48.272534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62655bfd-a8c0-4f97-9e12-4b4f93996198 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:48.272813 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62655bfd-a8c0-4f97-9e12-4b4f93996198 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:48.273083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62655bfd-a8c0-4f97-9e12-4b4f93996198 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:48.275395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62655bfd-a8c0-4f97-9e12-4b4f93996198 None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:48.276874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62655bfd-a8c0-4f97-9e12-4b4f93996198 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:48.278530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 166/346] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:48 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:48.281609 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d4b06-6cd8-4ea7-bacb-dfcff047a62d None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:48.281887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d4b06-6cd8-4ea7-bacb-dfcff047a62d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:48.282160 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d4b06-6cd8-4ea7-bacb-dfcff047a62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:48.305590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-bf9d4b06-6cd8-4ea7-bacb-dfcff047a62d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:48.307315 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bf9d4b06-6cd8-4ea7-bacb-dfcff047a62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:48.351471 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ababfe6-8dd0-4165-aa95-240bb8525a1c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:48.387799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:48.394337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 167/347] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:48.395873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e866697-48b8-4d9e-aa91-06b3e2c434b0 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:48.396315 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e866697-48b8-4d9e-aa91-06b3e2c434b0 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:48.396761 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e866697-48b8-4d9e-aa91-06b3e2c434b0 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:48.399438 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 168/348] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:48.406147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e866697-48b8-4d9e-aa91-06b3e2c434b0 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:48.408582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3d744-297a-4823-a34c-f5f9c9f733c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:48.409237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3d744-297a-4823-a34c-f5f9c9f733c0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:48.409518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3d744-297a-4823-a34c-f5f9c9f733c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:48.409792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3d744-297a-4823-a34c-f5f9c9f733c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:48.421584 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3d744-297a-4823-a34c-f5f9c9f733c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e9f3d744-297a-4823-a34c-f5f9c9f733c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:48.426625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:48.427552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:48.428504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:48.428504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:48.428504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:48.428504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:48.428504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:48.428504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:48.428504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:48.428504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:48.428504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:48.428504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:48.428504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:48.428504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:48.428504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 181/349] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:14:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:48.429181 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3d744-297a-4823-a34c-f5f9c9f733c0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:48.433440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:48.526251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e866697-48b8-4d9e-aa91-06b3e2c434b0 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:48.555676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3d744-297a-4823-a34c-f5f9c9f733c0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:48.563760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9848a-fa87-41a5-bf31-4c91b65816c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:48.564525 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9848a-fa87-41a5-bf31-4c91b65816c4 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:48.564910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9848a-fa87-41a5-bf31-4c91b65816c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:48.565349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9848a-fa87-41a5-bf31-4c91b65816c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:48.566116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9848a-fa87-41a5-bf31-4c91b65816c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:48.566525 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9848a-fa87-41a5-bf31-4c91b65816c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:48.568058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9848a-fa87-41a5-bf31-4c91b65816c4 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:48.581937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 169/350] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:48.583383 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07842e0-d0a7-4726-8794-68f8c591f015 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:48.583791 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07842e0-d0a7-4726-8794-68f8c591f015 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:48.584180 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07842e0-d0a7-4726-8794-68f8c591f015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:48.597853 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1c5c9-3ad2-4def-80b0-a00d5f0ceb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:48.598555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1c5c9-3ad2-4def-80b0-a00d5f0ceb8f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:48.598833 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1c5c9-3ad2-4def-80b0-a00d5f0ceb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:48.599110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1c5c9-3ad2-4def-80b0-a00d5f0ceb8f None admin] PATH_INFO: `/v3/groups/84cfc0496b5a4ea1bac088c831734672` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:48.606154 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b07842e0-d0a7-4726-8794-68f8c591f015 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:48.607403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b07842e0-d0a7-4726-8794-68f8c591f015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:48.608241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1c5c9-3ad2-4def-80b0-a00d5f0ceb8f None admin] RBAC: Authorizing `identity:get_group(group_id=84cfc0496b5a4ea1bac088c831734672)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:48.609967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1c5c9-3ad2-4def-80b0-a00d5f0ceb8f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:48.613347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 182/351] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:14:48 2022] GET /identity/v3/groups/84cfc0496b5a4ea1bac088c831734672 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:48.614221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1c5c9-3ad2-4def-80b0-a00d5f0ceb8f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:48.675250 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:48.684746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 170/352] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:48.685610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07842e0-d0a7-4726-8794-68f8c591f015 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:48.705641 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:48.758236 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1c5c9-3ad2-4def-80b0-a00d5f0ceb8f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:48.817461 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d2c33-5bf8-4230-b6d1-07b3a60e97ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:48.818098 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d2c33-5bf8-4230-b6d1-07b3a60e97ec None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:48.818433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d2c33-5bf8-4230-b6d1-07b3a60e97ec None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:48.818715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d2c33-5bf8-4230-b6d1-07b3a60e97ec None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:48.819391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d2c33-5bf8-4230-b6d1-07b3a60e97ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:48.819728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d2c33-5bf8-4230-b6d1-07b3a60e97ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:48.822482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d2c33-5bf8-4230-b6d1-07b3a60e97ec None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:48.834064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07842e0-d0a7-4726-8794-68f8c591f015 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:48.834369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 183/353] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:14:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:48.835237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d2c33-5bf8-4230-b6d1-07b3a60e97ec None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:48.886180 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f412558-2ef6-4d74-949d-a149379b1bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:48.886914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f412558-2ef6-4d74-949d-a149379b1bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:48.887266 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f412558-2ef6-4d74-949d-a149379b1bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:48.887652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f412558-2ef6-4d74-949d-a149379b1bd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:48.965747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d2c33-5bf8-4230-b6d1-07b3a60e97ec None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:48.966819 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:48.979346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f412558-2ef6-4d74-949d-a149379b1bd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:48.981256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f412558-2ef6-4d74-949d-a149379b1bd8 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:48.996066 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:49.004441 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea8114-d6c9-4d85-b1ad-a9936c7dec6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:49.005181 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea8114-d6c9-4d85-b1ad-a9936c7dec6c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:49.005465 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea8114-d6c9-4d85-b1ad-a9936c7dec6c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:49.005746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea8114-d6c9-4d85-b1ad-a9936c7dec6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:49.006480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea8114-d6c9-4d85-b1ad-a9936c7dec6c None admin] RBAC: Adding query filter params (group.id=84cfc0496b5a4ea1bac088c831734672, role.id=ebbe7434d1ec4bc4ba71737f55987dd9, scope.project.id=7849898da2494c4897ab5e0e09c9fe9e) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:49.006793 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea8114-d6c9-4d85-b1ad-a9936c7dec6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:49.010039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea8114-d6c9-4d85-b1ad-a9936c7dec6c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:49.017147 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 184/354] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Sep 12 23:14:48 2022] GET /identity/v3/role_assignments?group.id=84cfc0496b5a4ea1bac088c831734672&scope.project.id=7849898da2494c4897ab5e0e09c9fe9e&role.id=ebbe7434d1ec4bc4ba71737f55987dd9 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:49.018031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea8114-d6c9-4d85-b1ad-a9936c7dec6c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:49.105012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 171/355] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:49.105926 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f412558-2ef6-4d74-949d-a149379b1bd8 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:49.149519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea8114-d6c9-4d85-b1ad-a9936c7dec6c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:49.196868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b5a75-ce95-4e5f-95cd-a5baa53a5a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:49.197519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b5a75-ce95-4e5f-95cd-a5baa53a5a7b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:49.197802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b5a75-ce95-4e5f-95cd-a5baa53a5a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:49.198077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b5a75-ce95-4e5f-95cd-a5baa53a5a7b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:49.212816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b5a75-ce95-4e5f-95cd-a5baa53a5a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-999b5a75-ce95-4e5f-95cd-a5baa53a5a7b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:49.219343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:49.220385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:49.221171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:49.221171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:49.221171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:49.221171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:49.221171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:49.221171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:49.221171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:49.221171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:49.221171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:49.221171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:49.221171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:14:49.221171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:49.222992 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 185/356] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:49.223957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b5a75-ce95-4e5f-95cd-a5baa53a5a7b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:49.240510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:49.252188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f412558-2ef6-4d74-949d-a149379b1bd8 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:49.303445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:49.303913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad72b06-9011-47ae-b12b-da0f12794497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:49.304552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad72b06-9011-47ae-b12b-da0f12794497 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:49.304834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad72b06-9011-47ae-b12b-da0f12794497 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:49.305110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad72b06-9011-47ae-b12b-da0f12794497 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:49.373365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b5a75-ce95-4e5f-95cd-a5baa53a5a7b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:49.396640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad72b06-9011-47ae-b12b-da0f12794497 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:49.398478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad72b06-9011-47ae-b12b-da0f12794497 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:49.420665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb25cdd-1c25-400a-a009-d9ff748b74f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:49.421325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb25cdd-1c25-400a-a009-d9ff748b74f2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:49.421589 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb25cdd-1c25-400a-a009-d9ff748b74f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:49.421867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb25cdd-1c25-400a-a009-d9ff748b74f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:49.440890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb25cdd-1c25-400a-a009-d9ff748b74f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-acb25cdd-1c25-400a-a009-d9ff748b74f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:49.450259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:49.451199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:49.452069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:49.452069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:49.452069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:49.452069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:49.452069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:49.452069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:49.452069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:49.452069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:49.452069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:49.452069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:49.452069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:49.452069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:49.452690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 186/357] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:14:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:49.454485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107e9c1-87d0-475f-b3b5-0799fbd657a9 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:49.454485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107e9c1-87d0-475f-b3b5-0799fbd657a9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:49.454485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107e9c1-87d0-475f-b3b5-0799fbd657a9 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:49.455514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 187/358] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:49.456118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107e9c1-87d0-475f-b3b5-0799fbd657a9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:49.528701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:49.541818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 172/359] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:49.542662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad72b06-9011-47ae-b12b-da0f12794497 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:49.608153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107e9c1-87d0-475f-b3b5-0799fbd657a9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:49.624987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:49.652721 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e113f8-9be8-4c2f-a73a-39e5b3c0ee57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:49.653354 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e113f8-9be8-4c2f-a73a-39e5b3c0ee57 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:49.653629 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e113f8-9be8-4c2f-a73a-39e5b3c0ee57 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:49.653900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e113f8-9be8-4c2f-a73a-39e5b3c0ee57 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:49.655000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e113f8-9be8-4c2f-a73a-39e5b3c0ee57 None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:49.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e113f8-9be8-4c2f-a73a-39e5b3c0ee57 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:49.658670 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 188/360] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:49 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:49.659259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e113f8-9be8-4c2f-a73a-39e5b3c0ee57 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:49.675002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad72b06-9011-47ae-b12b-da0f12794497 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:49.719237 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072a17f-1b40-4b5c-afb3-1eae9f6d3eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:49.720140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072a17f-1b40-4b5c-afb3-1eae9f6d3eba None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:49.720649 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072a17f-1b40-4b5c-afb3-1eae9f6d3eba None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:49.721124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072a17f-1b40-4b5c-afb3-1eae9f6d3eba None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:49.722045 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072a17f-1b40-4b5c-afb3-1eae9f6d3eba None admin] RBAC: Adding query filter params (name=reader) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:49.722554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072a17f-1b40-4b5c-afb3-1eae9f6d3eba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:49.724308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072a17f-1b40-4b5c-afb3-1eae9f6d3eba None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:49.733995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 173/361] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:49.739668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb848d-1dab-4f5c-9163-b74f375d9c09 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:49.739668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb848d-1dab-4f5c-9163-b74f375d9c09 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:49.739668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb848d-1dab-4f5c-9163-b74f375d9c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:49.764414 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-66cb848d-1dab-4f5c-9163-b74f375d9c09 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:49.765629 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-66cb848d-1dab-4f5c-9163-b74f375d9c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:49.777637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e113f8-9be8-4c2f-a73a-39e5b3c0ee57 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:49.819391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a7502-e6ce-48f7-8a6d-7c24be95cad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:49.820222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a7502-e6ce-48f7-8a6d-7c24be95cad0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:49.820222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a7502-e6ce-48f7-8a6d-7c24be95cad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:49.820406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a7502-e6ce-48f7-8a6d-7c24be95cad0 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:49.821034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a7502-e6ce-48f7-8a6d-7c24be95cad0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:49.821337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a7502-e6ce-48f7-8a6d-7c24be95cad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:49.822825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a7502-e6ce-48f7-8a6d-7c24be95cad0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:49.828380 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:49.835021 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 189/362] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:14:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:49.839682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a7502-e6ce-48f7-8a6d-7c24be95cad0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:49.865453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 174/363] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:49.866884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb848d-1dab-4f5c-9163-b74f375d9c09 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:49.891086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:49.971968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a7502-e6ce-48f7-8a6d-7c24be95cad0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:50.012581 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb848d-1dab-4f5c-9163-b74f375d9c09 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:50.020396 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973aeafe-ae51-4d34-8d3c-471144de71c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:50.021044 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973aeafe-ae51-4d34-8d3c-471144de71c8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:50.021326 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973aeafe-ae51-4d34-8d3c-471144de71c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:50.021603 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973aeafe-ae51-4d34-8d3c-471144de71c8 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:50.023270 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973aeafe-ae51-4d34-8d3c-471144de71c8 None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:50.025341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973aeafe-ae51-4d34-8d3c-471144de71c8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:50.032011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 190/364] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:49 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:50.033094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973aeafe-ae51-4d34-8d3c-471144de71c8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:50.058446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e717cf-a888-4e65-834d-9dd1c1bce432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:50.059120 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e717cf-a888-4e65-834d-9dd1c1bce432 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:50.059400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e717cf-a888-4e65-834d-9dd1c1bce432 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:50.059711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e717cf-a888-4e65-834d-9dd1c1bce432 None admin] PATH_INFO: `/v3/groups/84cfc0496b5a4ea1bac088c831734672` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:50.061046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e717cf-a888-4e65-834d-9dd1c1bce432 None admin] RBAC: Authorizing `identity:get_group(group_id=84cfc0496b5a4ea1bac088c831734672)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:50.064476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e717cf-a888-4e65-834d-9dd1c1bce432 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:50.066132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 175/365] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:14:49 2022] GET /identity/v3/groups/84cfc0496b5a4ea1bac088c831734672 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:50.066888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e717cf-a888-4e65-834d-9dd1c1bce432 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:50.137842 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:50.156901 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973aeafe-ae51-4d34-8d3c-471144de71c8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:50.171643 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:50.199406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9b796-7766-4eaa-a773-d43d8ee0ee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:50.199695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e717cf-a888-4e65-834d-9dd1c1bce432 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:50.200167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9b796-7766-4eaa-a773-d43d8ee0ee21 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:50.200449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9b796-7766-4eaa-a773-d43d8ee0ee21 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:50.200714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9b796-7766-4eaa-a773-d43d8ee0ee21 None admin] PATH_INFO: `/v3/users/fcfa86e091284f6e8c291ed507390f0b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:50.202078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9b796-7766-4eaa-a773-d43d8ee0ee21 None admin] RBAC: Authorizing `identity:get_user(user_id=fcfa86e091284f6e8c291ed507390f0b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:50.204415 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9b796-7766-4eaa-a773-d43d8ee0ee21 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:50.206163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 191/366] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:14:50 2022] GET /identity/v3/users/fcfa86e091284f6e8c291ed507390f0b => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:50.206763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9b796-7766-4eaa-a773-d43d8ee0ee21 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:50.246812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3008fa-d00e-4abe-8a35-79bfb50cbcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:50.247430 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3008fa-d00e-4abe-8a35-79bfb50cbcf6 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:50.247738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3008fa-d00e-4abe-8a35-79bfb50cbcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:50.248010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3008fa-d00e-4abe-8a35-79bfb50cbcf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:50.338775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3008fa-d00e-4abe-8a35-79bfb50cbcf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:50.339891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9b796-7766-4eaa-a773-d43d8ee0ee21 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:50.340286 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3008fa-d00e-4abe-8a35-79bfb50cbcf6 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:50.382219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbaeb5-6184-4043-becc-579f767bf3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:50.383394 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbaeb5-6184-4043-becc-579f767bf3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:50.383394 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbaeb5-6184-4043-becc-579f767bf3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:50.383594 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbaeb5-6184-4043-becc-579f767bf3c5 None admin] PATH_INFO: `/v3/groups/84cfc0496b5a4ea1bac088c831734672` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:50.386077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbaeb5-6184-4043-becc-579f767bf3c5 None admin] RBAC: Authorizing `identity:get_group(group_id=84cfc0496b5a4ea1bac088c831734672)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:50.387012 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbaeb5-6184-4043-becc-579f767bf3c5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:50.390217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 192/367] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:14:50 2022] GET /identity/v3/groups/84cfc0496b5a4ea1bac088c831734672 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:50.391288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbaeb5-6184-4043-becc-579f767bf3c5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:50.411489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:50.455629 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:50.479680 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 176/368] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:50.480725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3008fa-d00e-4abe-8a35-79bfb50cbcf6 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:50.520849 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbaeb5-6184-4043-becc-579f767bf3c5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:50.562440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da674077-6623-4b8f-85d4-47c123dd50d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:50.563279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da674077-6623-4b8f-85d4-47c123dd50d3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:50.563384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da674077-6623-4b8f-85d4-47c123dd50d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:50.564437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da674077-6623-4b8f-85d4-47c123dd50d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:50.564718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da674077-6623-4b8f-85d4-47c123dd50d3 None admin] RBAC: Adding query filter params (group.id=84cfc0496b5a4ea1bac088c831734672, role.id=ebbe7434d1ec4bc4ba71737f55987dd9, scope.project.id=83919a658b6446d098540ce590417c23) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:50.565945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da674077-6623-4b8f-85d4-47c123dd50d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:50.567402 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da674077-6623-4b8f-85d4-47c123dd50d3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:50.575877 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 193/369] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Sep 12 23:14:50 2022] GET /identity/v3/role_assignments?group.id=84cfc0496b5a4ea1bac088c831734672&scope.project.id=83919a658b6446d098540ce590417c23&role.id=ebbe7434d1ec4bc4ba71737f55987dd9 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:50.576684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da674077-6623-4b8f-85d4-47c123dd50d3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:50.609377 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3008fa-d00e-4abe-8a35-79bfb50cbcf6 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:50.661400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701a297-bdf5-4bf1-956d-763151d041cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:50.662419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701a297-bdf5-4bf1-956d-763151d041cf None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:50.662869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701a297-bdf5-4bf1-956d-763151d041cf None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:50.663556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701a297-bdf5-4bf1-956d-763151d041cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:50.678947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701a297-bdf5-4bf1-956d-763151d041cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-c701a297-bdf5-4bf1-956d-763151d041cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:50.686410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:50.687549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:50.688573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:50.688573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:50.688573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:50.688573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:50.688573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:50.688573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:50.688573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:50.688573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:50.688573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:50.688573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:50.688573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:50.688573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:50.691793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 177/370] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:14:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:50.693110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701a297-bdf5-4bf1-956d-763151d041cf None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:50.710966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:50.720167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da674077-6623-4b8f-85d4-47c123dd50d3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:50.735134 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:50.767733 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b010caf-9ef6-4eb9-94f5-9d7fbfae3dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:50.768414 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b010caf-9ef6-4eb9-94f5-9d7fbfae3dbf None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:50.768700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b010caf-9ef6-4eb9-94f5-9d7fbfae3dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:50.768980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b010caf-9ef6-4eb9-94f5-9d7fbfae3dbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:50.769844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b010caf-9ef6-4eb9-94f5-9d7fbfae3dbf None admin] RBAC: Adding query filter params (role.id=c6bc2b1713e84fecb6708753f6ba6a66, user.id=fcfa86e091284f6e8c291ed507390f0b, scope.system=all) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:50.770161 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b010caf-9ef6-4eb9-94f5-9d7fbfae3dbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:50.771649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b010caf-9ef6-4eb9-94f5-9d7fbfae3dbf None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:50.786093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 194/371] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Sep 12 23:14:50 2022] GET /identity/v3/role_assignments?user.id=fcfa86e091284f6e8c291ed507390f0b&scope.system=all&role.id=c6bc2b1713e84fecb6708753f6ba6a66 => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:50.787280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b010caf-9ef6-4eb9-94f5-9d7fbfae3dbf None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:50.865985 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701a297-bdf5-4bf1-956d-763151d041cf None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:50.956402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53da6bb-b21c-47d8-94c2-9a5f728908cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:50.957032 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53da6bb-b21c-47d8-94c2-9a5f728908cf None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:50.957310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53da6bb-b21c-47d8-94c2-9a5f728908cf None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:50.957579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53da6bb-b21c-47d8-94c2-9a5f728908cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:50.958339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53da6bb-b21c-47d8-94c2-9a5f728908cf None admin] RBAC: Adding query filter params (group.id=84cfc0496b5a4ea1bac088c831734672, role.id=31058b9633b44d7ca64f103119987fd9, scope.project.id=7849898da2494c4897ab5e0e09c9fe9e) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:50.958644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53da6bb-b21c-47d8-94c2-9a5f728908cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:50.960220 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53da6bb-b21c-47d8-94c2-9a5f728908cf None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:50.974861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 178/372] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Sep 12 23:14:50 2022] GET /identity/v3/role_assignments?group.id=84cfc0496b5a4ea1bac088c831734672&scope.project.id=7849898da2494c4897ab5e0e09c9fe9e&role.id=31058b9633b44d7ca64f103119987fd9 => generated 260 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:50.975781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53da6bb-b21c-47d8-94c2-9a5f728908cf None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:50.976208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b010caf-9ef6-4eb9-94f5-9d7fbfae3dbf None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:51.021122 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2389ab6-ef15-4a86-9d46-8ed77f3efdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:51.021832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2389ab6-ef15-4a86-9d46-8ed77f3efdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:51.022121 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2389ab6-ef15-4a86-9d46-8ed77f3efdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:51.022402 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2389ab6-ef15-4a86-9d46-8ed77f3efdf8 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:51.023708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2389ab6-ef15-4a86-9d46-8ed77f3efdf8 None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:51.025352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2389ab6-ef15-4a86-9d46-8ed77f3efdf8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:51.027438 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 195/373] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:50 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:51.033379 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2389ab6-ef15-4a86-9d46-8ed77f3efdf8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:51.099121 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:51.126339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:51.169683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53da6bb-b21c-47d8-94c2-9a5f728908cf None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:51.194993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2389ab6-ef15-4a86-9d46-8ed77f3efdf8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:51.210435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0a824-5967-4209-ae07-f12f30c03bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:51.211447 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a824-5967-4209-ae07-f12f30c03bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:51.212047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a824-5967-4209-ae07-f12f30c03bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:51.213055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a824-5967-4209-ae07-f12f30c03bf0 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:51.213936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0a824-5967-4209-ae07-f12f30c03bf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:51.214446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0a824-5967-4209-ae07-f12f30c03bf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:51.216853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0a824-5967-4209-ae07-f12f30c03bf0 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:51.226176 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 179/374] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:51.234597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0a824-5967-4209-ae07-f12f30c03bf0 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:51.237078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2d72b-f537-4a72-8b9d-1cccb59c1291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:51.237925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2d72b-f537-4a72-8b9d-1cccb59c1291 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:51.238375 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2d72b-f537-4a72-8b9d-1cccb59c1291 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:51.238666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2d72b-f537-4a72-8b9d-1cccb59c1291 None admin] PATH_INFO: `/v3/groups/84cfc0496b5a4ea1bac088c831734672` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:51.240467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2d72b-f537-4a72-8b9d-1cccb59c1291 None admin] RBAC: Authorizing `identity:get_group(group_id=84cfc0496b5a4ea1bac088c831734672)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:51.241565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2d72b-f537-4a72-8b9d-1cccb59c1291 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:51.244196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 196/375] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:14:51 2022] GET /identity/v3/groups/84cfc0496b5a4ea1bac088c831734672 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:51.252208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2d72b-f537-4a72-8b9d-1cccb59c1291 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:51.385019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0a824-5967-4209-ae07-f12f30c03bf0 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:51.428062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e95ae-f115-4cde-aec7-10bcd974a4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:51.429366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e95ae-f115-4cde-aec7-10bcd974a4df None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:51.429789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e95ae-f115-4cde-aec7-10bcd974a4df None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:51.430194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e95ae-f115-4cde-aec7-10bcd974a4df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:51.432447 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2d72b-f537-4a72-8b9d-1cccb59c1291 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:51.435040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:51.441802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:51.515084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e95ae-f115-4cde-aec7-10bcd974a4df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:51.516679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e95ae-f115-4cde-aec7-10bcd974a4df None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:51.533156 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76543794-1a5f-4c7c-a6c7-d708798ca575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:51.533789 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76543794-1a5f-4c7c-a6c7-d708798ca575 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:51.534069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76543794-1a5f-4c7c-a6c7-d708798ca575 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:51.534333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76543794-1a5f-4c7c-a6c7-d708798ca575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:51.535070 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76543794-1a5f-4c7c-a6c7-d708798ca575 None admin] RBAC: Adding query filter params (group.id=84cfc0496b5a4ea1bac088c831734672, role.id=31058b9633b44d7ca64f103119987fd9, scope.project.id=83919a658b6446d098540ce590417c23) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:51.535381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76543794-1a5f-4c7c-a6c7-d708798ca575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:51.536851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76543794-1a5f-4c7c-a6c7-d708798ca575 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:51.548425 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 197/376] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Sep 12 23:14:51 2022] GET /identity/v3/role_assignments?group.id=84cfc0496b5a4ea1bac088c831734672&scope.project.id=83919a658b6446d098540ce590417c23&role.id=31058b9633b44d7ca64f103119987fd9 => generated 260 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:51.554667 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bba43f-de3e-429b-84d8-3f948d8ed1e7 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:51.554966 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bba43f-de3e-429b-84d8-3f948d8ed1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:51.555229 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bba43f-de3e-429b-84d8-3f948d8ed1e7 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:51.557373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 198/377] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:14:51.566888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dcf20-6eec-4e21-92e6-e15192f8c55f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:51.567197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dcf20-6eec-4e21-92e6-e15192f8c55f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:51.567448 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dcf20-6eec-4e21-92e6-e15192f8c55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:51.600832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-171dcf20-6eec-4e21-92e6-e15192f8c55f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:51.602205 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-171dcf20-6eec-4e21-92e6-e15192f8c55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:51.647665 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 180/378] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:14:51.653389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e95ae-f115-4cde-aec7-10bcd974a4df None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:51.716013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:51.758869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 199/379] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:51.799405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e95ae-f115-4cde-aec7-10bcd974a4df None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:51.834744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388407a-8e13-4dd3-8864-7e527ebe46ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:51.835379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388407a-8e13-4dd3-8864-7e527ebe46ff None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:51.835694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388407a-8e13-4dd3-8864-7e527ebe46ff None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:51.835976 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388407a-8e13-4dd3-8864-7e527ebe46ff None admin] PATH_INFO: `/v3/projects/5b515d316021473bab5135bffb78aaff` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:51.837136 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388407a-8e13-4dd3-8864-7e527ebe46ff None admin] RBAC: Authorizing `identity:get_project(project_id=5b515d316021473bab5135bffb78aaff)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:51.838629 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388407a-8e13-4dd3-8864-7e527ebe46ff None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:51.840663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 181/380] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:51 2022] GET /identity/v3/projects/5b515d316021473bab5135bffb78aaff => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:51.846413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dcf20-6eec-4e21-92e6-e15192f8c55f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:51.964943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:51.969186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dcf20-6eec-4e21-92e6-e15192f8c55f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:52.006773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608dd00-c058-4b53-bc8d-5a3ec784c412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:52.007420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608dd00-c058-4b53-bc8d-5a3ec784c412 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:52.007752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608dd00-c058-4b53-bc8d-5a3ec784c412 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:52.008032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608dd00-c058-4b53-bc8d-5a3ec784c412 None admin] PATH_INFO: `/v3/groups/9c7caa2d948e42c1ae1bc416139e40ed` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:52.015336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608dd00-c058-4b53-bc8d-5a3ec784c412 None admin] RBAC: Authorizing `identity:get_group(group_id=9c7caa2d948e42c1ae1bc416139e40ed)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:52.016957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608dd00-c058-4b53-bc8d-5a3ec784c412 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:52.018930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 200/381] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:14:51 2022] GET /identity/v3/groups/9c7caa2d948e42c1ae1bc416139e40ed => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:52.025241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388407a-8e13-4dd3-8864-7e527ebe46ff None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:52.141386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388407a-8e13-4dd3-8864-7e527ebe46ff None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:52.143681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:52.183071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd21532-ce55-4d41-b337-20aade519165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:52.183717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd21532-ce55-4d41-b337-20aade519165 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:52.183993 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd21532-ce55-4d41-b337-20aade519165 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:52.184262 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd21532-ce55-4d41-b337-20aade519165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:52.185097 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd21532-ce55-4d41-b337-20aade519165 None admin] RBAC: Adding query filter params (group.id=9c7caa2d948e42c1ae1bc416139e40ed, role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=5b515d316021473bab5135bffb78aaff) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:52.185408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd21532-ce55-4d41-b337-20aade519165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:52.186831 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd21532-ce55-4d41-b337-20aade519165 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:52.192917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 182/382] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Sep 12 23:14:52 2022] GET /identity/v3/role_assignments?group.id=9c7caa2d948e42c1ae1bc416139e40ed&scope.project.id=5b515d316021473bab5135bffb78aaff&role.id=23538dc4c770454e80dd11ed8e498539 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:52.266651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608dd00-c058-4b53-bc8d-5a3ec784c412 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:52.388473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608dd00-c058-4b53-bc8d-5a3ec784c412 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:52.426425 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb70794-0f17-4ecc-8d93-525cb3051d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:52.427081 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb70794-0f17-4ecc-8d93-525cb3051d26 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:52.427359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb70794-0f17-4ecc-8d93-525cb3051d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:52.427657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb70794-0f17-4ecc-8d93-525cb3051d26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:52.440280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb70794-0f17-4ecc-8d93-525cb3051d26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-fbb70794-0f17-4ecc-8d93-525cb3051d26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:52.445381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:52.446315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:52.447169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:52.447169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:52.447169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:52.447169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:52.447169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:52.447169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:52.447169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:52.447169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:52.447169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:52.447169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:52.447169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:52.447169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:52.447169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 201/383] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:14:52.452768 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd21532-ce55-4d41-b337-20aade519165 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:52.590640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd21532-ce55-4d41-b337-20aade519165 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:52.636857 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1433a-791f-4055-a4e6-671b4c9e3656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:52.637474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1433a-791f-4055-a4e6-671b4c9e3656 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:52.637755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1433a-791f-4055-a4e6-671b4c9e3656 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:52.638027 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1433a-791f-4055-a4e6-671b4c9e3656 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:52.638692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1433a-791f-4055-a4e6-671b4c9e3656 None admin] RBAC: Adding query filter params (name=member) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:52.638994 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1433a-791f-4055-a4e6-671b4c9e3656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:52.640394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1433a-791f-4055-a4e6-671b4c9e3656 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:52.655322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 183/384] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:52.661024 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb70794-0f17-4ecc-8d93-525cb3051d26 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:52.725423 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:52.850467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb70794-0f17-4ecc-8d93-525cb3051d26 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:52.904249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb4331-e3cc-4641-a1e1-d56f4ca596fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:52.904939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb4331-e3cc-4641-a1e1-d56f4ca596fb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:52.905223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb4331-e3cc-4641-a1e1-d56f4ca596fb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:52.905539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb4331-e3cc-4641-a1e1-d56f4ca596fb None admin] PATH_INFO: `/v3/groups/84cfc0496b5a4ea1bac088c831734672` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:52.908545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb4331-e3cc-4641-a1e1-d56f4ca596fb None admin] RBAC: Authorizing `identity:get_group(group_id=84cfc0496b5a4ea1bac088c831734672)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:52.910011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb4331-e3cc-4641-a1e1-d56f4ca596fb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:52.913234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 202/385] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:14:52 2022] GET /identity/v3/groups/84cfc0496b5a4ea1bac088c831734672 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:52.924225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1433a-791f-4055-a4e6-671b4c9e3656 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:52.968426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:53.067921 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1433a-791f-4055-a4e6-671b4c9e3656 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:53.123007 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3306fee-d6d2-42a1-893f-4b88b530e04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:53.123717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3306fee-d6d2-42a1-893f-4b88b530e04d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:53.124081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3306fee-d6d2-42a1-893f-4b88b530e04d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:53.124360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3306fee-d6d2-42a1-893f-4b88b530e04d None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:53.125600 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3306fee-d6d2-42a1-893f-4b88b530e04d None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:53.127093 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3306fee-d6d2-42a1-893f-4b88b530e04d None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:53.128859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 184/386] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:52 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:53.135507 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb4331-e3cc-4641-a1e1-d56f4ca596fb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:53.233851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af19727-a922-4289-b81c-96f7af4961dd None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:53.234232 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af19727-a922-4289-b81c-96f7af4961dd None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:53.234392 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af19727-a922-4289-b81c-96f7af4961dd None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:53.235809 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 185/387] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:53.241257 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0bfb1-e3ec-4cbb-8858-bc13d6715a6f None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:53.241540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0bfb1-e3ec-4cbb-8858-bc13d6715a6f None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:53.241802 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0bfb1-e3ec-4cbb-8858-bc13d6715a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:53.251278 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:53.263123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-e2a0bfb1-e3ec-4cbb-8858-bc13d6715a6f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:53.264520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e2a0bfb1-e3ec-4cbb-8858-bc13d6715a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:53.267210 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb4331-e3cc-4641-a1e1-d56f4ca596fb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:53.329063 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b8595-e10a-4cbc-af71-0ed8f6280317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:53.329660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b8595-e10a-4cbc-af71-0ed8f6280317 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:53.329943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b8595-e10a-4cbc-af71-0ed8f6280317 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:53.330225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b8595-e10a-4cbc-af71-0ed8f6280317 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e/groups/84cfc0496b5a4ea1bac088c831734672/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:53.337104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b8595-e10a-4cbc-af71-0ed8f6280317 None admin] RBAC: Authorizing `identity:create_grant(project_id=7849898da2494c4897ab5e0e09c9fe9e, group_id=84cfc0496b5a4ea1bac088c831734672, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:53.338634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b8595-e10a-4cbc-af71-0ed8f6280317 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:53.358020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 203/388] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Sep 12 23:14:53 2022] PUT /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e/groups/84cfc0496b5a4ea1bac088c831734672/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:14:53.386850 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 186/389] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:53.716703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b8595-e10a-4cbc-af71-0ed8f6280317 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:53.847175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b8595-e10a-4cbc-af71-0ed8f6280317 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:53.891685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cabb4-de2e-49fd-996a-b23846246af1 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:53.891685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cabb4-de2e-49fd-996a-b23846246af1 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:53.891685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cabb4-de2e-49fd-996a-b23846246af1 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:53.893965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 187/390] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:53.898449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed5833-77f5-4217-ba09-7c38e077f032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:53.899093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed5833-77f5-4217-ba09-7c38e077f032 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:53.899381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed5833-77f5-4217-ba09-7c38e077f032 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:53.899761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed5833-77f5-4217-ba09-7c38e077f032 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:53.900298 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29457ac5-b58a-48d7-a4b5-4eaf791e6922 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:53.900824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29457ac5-b58a-48d7-a4b5-4eaf791e6922 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:53.901283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29457ac5-b58a-48d7-a4b5-4eaf791e6922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:53.916578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed5833-77f5-4217-ba09-7c38e077f032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e6ed5833-77f5-4217-ba09-7c38e077f032 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:53.921947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:53.922825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:53.923639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:53.923639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:53.923639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:53.923639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:53.923639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:53.923639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:53.923639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:53.923639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:53.923639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:53.923639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:53.923639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:53.923639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:53.923639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 204/391] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:53.929792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed5833-77f5-4217-ba09-7c38e077f032 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:53.935088 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-29457ac5-b58a-48d7-a4b5-4eaf791e6922 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:53.936540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-29457ac5-b58a-48d7-a4b5-4eaf791e6922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:53.949619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:53.990296 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:54.044193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed5833-77f5-4217-ba09-7c38e077f032 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:54.077643 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 188/392] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:54.112241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c600a-8cf3-4dbd-88bd-93f349f90563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:54.112855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c600a-8cf3-4dbd-88bd-93f349f90563 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:54.113134 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c600a-8cf3-4dbd-88bd-93f349f90563 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:54.113403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c600a-8cf3-4dbd-88bd-93f349f90563 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:54.114063 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c600a-8cf3-4dbd-88bd-93f349f90563 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:54.114366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c600a-8cf3-4dbd-88bd-93f349f90563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:54.115750 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c600a-8cf3-4dbd-88bd-93f349f90563 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:54.131025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 205/393] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:54.137500 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29457ac5-b58a-48d7-a4b5-4eaf791e6922 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:54.282474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29457ac5-b58a-48d7-a4b5-4eaf791e6922 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:54.318556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300644b-5fda-43b1-82b5-e59b02cbe354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:54.319524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300644b-5fda-43b1-82b5-e59b02cbe354 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:54.320039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300644b-5fda-43b1-82b5-e59b02cbe354 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:54.320501 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300644b-5fda-43b1-82b5-e59b02cbe354 None admin] PATH_INFO: `/v3/groups/84cfc0496b5a4ea1bac088c831734672` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:54.321914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300644b-5fda-43b1-82b5-e59b02cbe354 None admin] RBAC: Authorizing `identity:get_group(group_id=84cfc0496b5a4ea1bac088c831734672)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:54.323540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300644b-5fda-43b1-82b5-e59b02cbe354 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:54.325335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 189/394] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:14:54 2022] GET /identity/v3/groups/84cfc0496b5a4ea1bac088c831734672 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:54.331066 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c600a-8cf3-4dbd-88bd-93f349f90563 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:54.420189 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:54.456056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c600a-8cf3-4dbd-88bd-93f349f90563 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:54.457391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300644b-5fda-43b1-82b5-e59b02cbe354 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:54.486526 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:54.502544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd65caa-2d16-464d-b2bb-8aaf2dabe4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:54.503518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd65caa-2d16-464d-b2bb-8aaf2dabe4a7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:54.504003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd65caa-2d16-464d-b2bb-8aaf2dabe4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:54.504481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd65caa-2d16-464d-b2bb-8aaf2dabe4a7 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:54.505796 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd65caa-2d16-464d-b2bb-8aaf2dabe4a7 None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:54.507396 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd65caa-2d16-464d-b2bb-8aaf2dabe4a7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:54.509690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 206/395] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:54 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:54.519838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd65caa-2d16-464d-b2bb-8aaf2dabe4a7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:54.587799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300644b-5fda-43b1-82b5-e59b02cbe354 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:54.648412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd65caa-2d16-464d-b2bb-8aaf2dabe4a7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:54.662563 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cf19a-809d-40df-8fb2-1bed8f26c3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:54.663243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cf19a-809d-40df-8fb2-1bed8f26c3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:54.663589 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cf19a-809d-40df-8fb2-1bed8f26c3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:54.663868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cf19a-809d-40df-8fb2-1bed8f26c3b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:54.676776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08cf19a-809d-40df-8fb2-1bed8f26c3b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-c08cf19a-809d-40df-8fb2-1bed8f26c3b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:54.681752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:54.682653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:54.683446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:54.683446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:54.683446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:54.683446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:54.683446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:54.683446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:54.683446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:54.683446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:54.683446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:54.683446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:54.683446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:54.683446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:54.683446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 190/396] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:14:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:54.685017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b8df3-0b06-48f4-b072-e247dde2f8a5 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:54.685304 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b8df3-0b06-48f4-b072-e247dde2f8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:54.685581 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b8df3-0b06-48f4-b072-e247dde2f8a5 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:54.690867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 191/397] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:54.690867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b8df3-0b06-48f4-b072-e247dde2f8a5 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:54.702927 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b451215-980b-487d-9c55-470110d78cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:54.704330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b451215-980b-487d-9c55-470110d78cc5 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:54.704913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b451215-980b-487d-9c55-470110d78cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:54.705383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b451215-980b-487d-9c55-470110d78cc5 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23/groups/84cfc0496b5a4ea1bac088c831734672/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:54.707504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b451215-980b-487d-9c55-470110d78cc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=83919a658b6446d098540ce590417c23, group_id=84cfc0496b5a4ea1bac088c831734672, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:54.709425 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b451215-980b-487d-9c55-470110d78cc5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:54.721358 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:54.736052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 207/398] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Sep 12 23:14:54 2022] PUT /identity/v3/projects/83919a658b6446d098540ce590417c23/groups/84cfc0496b5a4ea1bac088c831734672/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:14:54.737815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce569065-3298-40bd-870d-98d459766fcb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:54.738686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce569065-3298-40bd-870d-98d459766fcb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:54.739183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce569065-3298-40bd-870d-98d459766fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:54.788821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ce569065-3298-40bd-870d-98d459766fcb None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:54.790090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ce569065-3298-40bd-870d-98d459766fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:54.808723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:54.830596 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b8df3-0b06-48f4-b072-e247dde2f8a5 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:54.874145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9253a0-3f36-4552-87a3-5a82352e15e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:54.875216 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9253a0-3f36-4552-87a3-5a82352e15e5 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:54.875766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9253a0-3f36-4552-87a3-5a82352e15e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:54.876230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9253a0-3f36-4552-87a3-5a82352e15e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:54.877123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9253a0-3f36-4552-87a3-5a82352e15e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:54.877673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9253a0-3f36-4552-87a3-5a82352e15e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:54.879298 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9253a0-3f36-4552-87a3-5a82352e15e5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:54.897229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 192/399] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:14:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:54.898520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf9556-615b-4585-aaff-98b1cc540f16 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:54.898808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf9556-615b-4585-aaff-98b1cc540f16 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:54.899372 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf9556-615b-4585-aaff-98b1cc540f16 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:54.900998 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 193/400] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:54.904359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf9556-615b-4585-aaff-98b1cc540f16 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:54.951746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 208/401] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:54.954397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e88c91-bb0d-4c06-84c7-e95accdc9178 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:54.954896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e88c91-bb0d-4c06-84c7-e95accdc9178 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:54.955347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e88c91-bb0d-4c06-84c7-e95accdc9178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:54.979946 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c3e88c91-bb0d-4c06-84c7-e95accdc9178 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:54.981456 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c3e88c91-bb0d-4c06-84c7-e95accdc9178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:55.054908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf9556-615b-4585-aaff-98b1cc540f16 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:55.065616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 209/402] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:55.103930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1a22b-aec4-4098-a047-ba54a87e2d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:55.104569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1a22b-aec4-4098-a047-ba54a87e2d0c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:55.104904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1a22b-aec4-4098-a047-ba54a87e2d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:55.105174 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1a22b-aec4-4098-a047-ba54a87e2d0c None admin] PATH_INFO: `/v3/groups/84cfc0496b5a4ea1bac088c831734672` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:55.106284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a1a22b-aec4-4098-a047-ba54a87e2d0c None admin] RBAC: Authorizing `identity:get_group(group_id=84cfc0496b5a4ea1bac088c831734672)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:55.107830 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a1a22b-aec4-4098-a047-ba54a87e2d0c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:55.109652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 194/403] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:14:54 2022] GET /identity/v3/groups/84cfc0496b5a4ea1bac088c831734672 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:55.115229 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e88c91-bb0d-4c06-84c7-e95accdc9178 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:55.190265 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:55.237525 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e88c91-bb0d-4c06-84c7-e95accdc9178 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:55.239839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1a22b-aec4-4098-a047-ba54a87e2d0c None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:55.268250 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:55.313595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d927f48-bf31-44bf-95d7-5871dd72c7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:55.314232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d927f48-bf31-44bf-95d7-5871dd72c7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:55.314510 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d927f48-bf31-44bf-95d7-5871dd72c7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:55.314781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d927f48-bf31-44bf-95d7-5871dd72c7f6 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:55.318245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d927f48-bf31-44bf-95d7-5871dd72c7f6 None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:55.319691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d927f48-bf31-44bf-95d7-5871dd72c7f6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:55.322238 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 210/404] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:55 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:55.327437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d927f48-bf31-44bf-95d7-5871dd72c7f6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:55.448085 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1a22b-aec4-4098-a047-ba54a87e2d0c None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:55.471086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d927f48-bf31-44bf-95d7-5871dd72c7f6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:55.505025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:55.514969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55874d6b-9718-4e22-b685-a35c3018fe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:55.515832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55874d6b-9718-4e22-b685-a35c3018fe52 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:55.515972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b04bc-010b-41e2-b7b5-a25e1285d709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:55.516391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55874d6b-9718-4e22-b685-a35c3018fe52 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:55.516775 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b04bc-010b-41e2-b7b5-a25e1285d709 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:55.516952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55874d6b-9718-4e22-b685-a35c3018fe52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:55.517225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b04bc-010b-41e2-b7b5-a25e1285d709 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:55.517801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b04bc-010b-41e2-b7b5-a25e1285d709 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e/groups/84cfc0496b5a4ea1bac088c831734672/roles/31058b9633b44d7ca64f103119987fd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:55.520810 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b04bc-010b-41e2-b7b5-a25e1285d709 None admin] RBAC: Authorizing `identity:create_grant(project_id=7849898da2494c4897ab5e0e09c9fe9e, group_id=84cfc0496b5a4ea1bac088c831734672, role_id=31058b9633b44d7ca64f103119987fd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:55.522526 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b04bc-010b-41e2-b7b5-a25e1285d709 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:55.530345 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55874d6b-9718-4e22-b685-a35c3018fe52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-55874d6b-9718-4e22-b685-a35c3018fe52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:55.536889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:55.537840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:55.538648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:55.538648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:55.538648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:55.538648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:55.538648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:55.538648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:55.538648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:55.538648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:55.538648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:55.538648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:55.538648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:55.538648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:55.539254 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 211/405] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Sep 12 23:14:55 2022] PUT /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e/groups/84cfc0496b5a4ea1bac088c831734672/roles/31058b9633b44d7ca64f103119987fd9 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:14:55.541557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b04bc-010b-41e2-b7b5-a25e1285d709 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:55.542324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 195/406] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:14:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:14:55.553496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55874d6b-9718-4e22-b685-a35c3018fe52 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:55.662847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b04bc-010b-41e2-b7b5-a25e1285d709 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:55.702847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59d097-140f-4020-bfbe-416a6c9d691f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:55.703427 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59d097-140f-4020-bfbe-416a6c9d691f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:55.703734 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59d097-140f-4020-bfbe-416a6c9d691f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:55.704007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59d097-140f-4020-bfbe-416a6c9d691f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:55.716499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59d097-140f-4020-bfbe-416a6c9d691f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-3e59d097-140f-4020-bfbe-416a6c9d691f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:55.721616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:55.722586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:55.723373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:55.723373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:55.723373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:55.723373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:55.723373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:55.723373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:55.723373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:55.723373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:55.723373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:55.723373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:55.723373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:55.723373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:55.723373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 212/407] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:14:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:55.727775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:55.728435 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59d097-140f-4020-bfbe-416a6c9d691f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:55.761497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55874d6b-9718-4e22-b685-a35c3018fe52 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:55.789530 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:55.807296 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713f9e5-4a8a-4dd3-8fb0-a2034e0ad413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:55.808247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713f9e5-4a8a-4dd3-8fb0-a2034e0ad413 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:55.808544 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713f9e5-4a8a-4dd3-8fb0-a2034e0ad413 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:55.808877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713f9e5-4a8a-4dd3-8fb0-a2034e0ad413 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:55.809828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713f9e5-4a8a-4dd3-8fb0-a2034e0ad413 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:55.810132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713f9e5-4a8a-4dd3-8fb0-a2034e0ad413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:55.811602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713f9e5-4a8a-4dd3-8fb0-a2034e0ad413 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:55.823230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 196/408] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:14:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:55.828711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713f9e5-4a8a-4dd3-8fb0-a2034e0ad413 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:55.859827 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59d097-140f-4020-bfbe-416a6c9d691f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:55.923668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7743a75-0768-43c3-9295-28d129252864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:55.926091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7743a75-0768-43c3-9295-28d129252864 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:55.926840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7743a75-0768-43c3-9295-28d129252864 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:55.928568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7743a75-0768-43c3-9295-28d129252864 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:55.929524 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7743a75-0768-43c3-9295-28d129252864 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:55.930259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7743a75-0768-43c3-9295-28d129252864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:55.933483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7743a75-0768-43c3-9295-28d129252864 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:55.950948 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 213/409] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:55.958537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7743a75-0768-43c3-9295-28d129252864 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:56.009012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713f9e5-4a8a-4dd3-8fb0-a2034e0ad413 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:56.089263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7743a75-0768-43c3-9295-28d129252864 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:56.093377 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2b27d-7cc3-4cba-9ff0-9686d535d885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:56.094028 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2b27d-7cc3-4cba-9ff0-9686d535d885 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:56.094314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2b27d-7cc3-4cba-9ff0-9686d535d885 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:56.094594 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2b27d-7cc3-4cba-9ff0-9686d535d885 None admin] PATH_INFO: `/v3/groups/84cfc0496b5a4ea1bac088c831734672` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:56.095762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2b27d-7cc3-4cba-9ff0-9686d535d885 None admin] RBAC: Authorizing `identity:get_group(group_id=84cfc0496b5a4ea1bac088c831734672)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:56.097209 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2b27d-7cc3-4cba-9ff0-9686d535d885 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:56.099099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 197/410] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:14:55 2022] GET /identity/v3/groups/84cfc0496b5a4ea1bac088c831734672 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:56.100376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f7a1a-2dd3-4cf9-ae66-9f93b495d2d4 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:56.100690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f7a1a-2dd3-4cf9-ae66-9f93b495d2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:56.101175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f7a1a-2dd3-4cf9-ae66-9f93b495d2d4 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:56.102398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 198/411] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:56.103436 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:56.108077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f7a1a-2dd3-4cf9-ae66-9f93b495d2d4 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:56.117405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:56.133830 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060315e-c83c-4901-9d10-e566595aedd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:56.135345 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060315e-c83c-4901-9d10-e566595aedd5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:56.135887 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060315e-c83c-4901-9d10-e566595aedd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:56.136355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060315e-c83c-4901-9d10-e566595aedd5 None admin] PATH_INFO: `/v3/groups/9c7caa2d948e42c1ae1bc416139e40ed` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:56.137806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060315e-c83c-4901-9d10-e566595aedd5 None admin] RBAC: Authorizing `identity:get_group(group_id=9c7caa2d948e42c1ae1bc416139e40ed)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:56.139634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060315e-c83c-4901-9d10-e566595aedd5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:56.141870 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 214/412] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:14:55 2022] GET /identity/v3/groups/9c7caa2d948e42c1ae1bc416139e40ed => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:56.143687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a0709-6f5a-4a35-8e9b-72b933593920 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:56.147976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a0709-6f5a-4a35-8e9b-72b933593920 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:56.148414 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a0709-6f5a-4a35-8e9b-72b933593920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:56.168011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-897a0709-6f5a-4a35-8e9b-72b933593920 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:56.171442 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-897a0709-6f5a-4a35-8e9b-72b933593920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:56.219403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f7a1a-2dd3-4cf9-ae66-9f93b495d2d4 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:56.247335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 215/413] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:56.248347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a0709-6f5a-4a35-8e9b-72b933593920 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:56.255889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c2af7-3cb0-4a79-a9ce-eb505bebaec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:56.256749 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c2af7-3cb0-4a79-a9ce-eb505bebaec5 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:56.257200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c2af7-3cb0-4a79-a9ce-eb505bebaec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:56.257668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c2af7-3cb0-4a79-a9ce-eb505bebaec5 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:56.258961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c2af7-3cb0-4a79-a9ce-eb505bebaec5 None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:56.260927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c2af7-3cb0-4a79-a9ce-eb505bebaec5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:56.263208 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 199/414] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:56 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:56.272621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c2af7-3cb0-4a79-a9ce-eb505bebaec5 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:56.351944 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a0709-6f5a-4a35-8e9b-72b933593920 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:56.362061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:56.391322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bea60-3fb0-4f5f-9841-793850a29f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:56.392577 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bea60-3fb0-4f5f-9841-793850a29f39 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:56.393052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bea60-3fb0-4f5f-9841-793850a29f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:56.393542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bea60-3fb0-4f5f-9841-793850a29f39 None admin] PATH_INFO: `/v3/projects/5b515d316021473bab5135bffb78aaff` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:56.395156 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bea60-3fb0-4f5f-9841-793850a29f39 None admin] RBAC: Authorizing `identity:get_project(project_id=5b515d316021473bab5135bffb78aaff)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:56.396820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bea60-3fb0-4f5f-9841-793850a29f39 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:56.398655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 216/415] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:56 2022] GET /identity/v3/projects/5b515d316021473bab5135bffb78aaff => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:56.405242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bea60-3fb0-4f5f-9841-793850a29f39 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:56.411476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:56.490152 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c2af7-3cb0-4a79-a9ce-eb505bebaec5 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:56.518041 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bea60-3fb0-4f5f-9841-793850a29f39 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:56.546531 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e799b-488f-4ea0-beb6-6b84a584401e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:56.547184 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e799b-488f-4ea0-beb6-6b84a584401e None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:56.547471 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e799b-488f-4ea0-beb6-6b84a584401e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:56.547775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e799b-488f-4ea0-beb6-6b84a584401e None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23/groups/84cfc0496b5a4ea1bac088c831734672/roles/31058b9633b44d7ca64f103119987fd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:56.549947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e799b-488f-4ea0-beb6-6b84a584401e None admin] RBAC: Authorizing `identity:create_grant(project_id=83919a658b6446d098540ce590417c23, group_id=84cfc0496b5a4ea1bac088c831734672, role_id=31058b9633b44d7ca64f103119987fd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:56.551406 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e799b-488f-4ea0-beb6-6b84a584401e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:56.561155 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262439b9-5553-4859-9c6d-268300bcaa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:56.562361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262439b9-5553-4859-9c6d-268300bcaa51 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:56.563086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262439b9-5553-4859-9c6d-268300bcaa51 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:56.567424 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262439b9-5553-4859-9c6d-268300bcaa51 None admin] PATH_INFO: `/v3/projects/5b515d316021473bab5135bffb78aaff/groups/9c7caa2d948e42c1ae1bc416139e40ed/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:56.567617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 200/416] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Sep 12 23:14:56 2022] PUT /identity/v3/projects/83919a658b6446d098540ce590417c23/groups/84cfc0496b5a4ea1bac088c831734672/roles/31058b9633b44d7ca64f103119987fd9 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:14:56.568742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e799b-488f-4ea0-beb6-6b84a584401e None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:56.570817 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262439b9-5553-4859-9c6d-268300bcaa51 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b515d316021473bab5135bffb78aaff, group_id=9c7caa2d948e42c1ae1bc416139e40ed, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:56.576855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262439b9-5553-4859-9c6d-268300bcaa51 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:56.603855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 217/417] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Sep 12 23:14:56 2022] PUT /identity/v3/projects/5b515d316021473bab5135bffb78aaff/groups/9c7caa2d948e42c1ae1bc416139e40ed/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:14:56.770841 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e799b-488f-4ea0-beb6-6b84a584401e None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:56.807676 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322113ed-5d0f-4f54-990e-903b0cb73e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:56.808395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322113ed-5d0f-4f54-990e-903b0cb73e98 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:56.808681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322113ed-5d0f-4f54-990e-903b0cb73e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:56.808957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322113ed-5d0f-4f54-990e-903b0cb73e98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:56.811877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:56.820194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322113ed-5d0f-4f54-990e-903b0cb73e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-322113ed-5d0f-4f54-990e-903b0cb73e98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:56.827554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:56.828519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:56.829328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:56.829328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:56.829328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:56.829328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:56.829328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:56.829328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:56.829328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:56.829328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:56.829328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:56.829328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:56.829328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:56.829328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:56.832150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 201/418] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:56.837736 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262439b9-5553-4859-9c6d-268300bcaa51 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:56.873856 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:56.945251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262439b9-5553-4859-9c6d-268300bcaa51 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:56.985726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec00fbf-f769-446d-b541-4e6aa0d7632d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:56.986348 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec00fbf-f769-446d-b541-4e6aa0d7632d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:56.986617 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec00fbf-f769-446d-b541-4e6aa0d7632d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:56.986885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec00fbf-f769-446d-b541-4e6aa0d7632d None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:56.987549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec00fbf-f769-446d-b541-4e6aa0d7632d None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:56.987869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec00fbf-f769-446d-b541-4e6aa0d7632d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:56.989225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec00fbf-f769-446d-b541-4e6aa0d7632d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:57.004002 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 218/419] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:57.004546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322113ed-5d0f-4f54-990e-903b0cb73e98 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:57.135027 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322113ed-5d0f-4f54-990e-903b0cb73e98 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:57.184064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe496aa5-6ff3-4266-8121-656a54851c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:57.185029 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe496aa5-6ff3-4266-8121-656a54851c84 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:57.185510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe496aa5-6ff3-4266-8121-656a54851c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:57.185955 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe496aa5-6ff3-4266-8121-656a54851c84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:57.257621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:57.269627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe496aa5-6ff3-4266-8121-656a54851c84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:57.271362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe496aa5-6ff3-4266-8121-656a54851c84 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:57.390686 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 202/420] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:57.395991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec00fbf-f769-446d-b541-4e6aa0d7632d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:57.518130 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:57.525460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec00fbf-f769-446d-b541-4e6aa0d7632d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:57.581254 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384f728-49ed-490e-815d-2e03a777bac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:57.581890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384f728-49ed-490e-815d-2e03a777bac9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:57.582170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384f728-49ed-490e-815d-2e03a777bac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:57.582445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384f728-49ed-490e-815d-2e03a777bac9 None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:57.583597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384f728-49ed-490e-815d-2e03a777bac9 None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:57.585025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384f728-49ed-490e-815d-2e03a777bac9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:57.586645 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 219/421] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:57 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:57.592507 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe496aa5-6ff3-4266-8121-656a54851c84 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:57.641350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55133b4-2ca0-41b8-967c-252fa7726c02 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:57.641554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55133b4-2ca0-41b8-967c-252fa7726c02 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:57.641832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55133b4-2ca0-41b8-967c-252fa7726c02 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:57.643154 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 220/422] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:14:57.654537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ade6-cbdc-428e-95f2-b63c3d1e895f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:57.654817 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ade6-cbdc-428e-95f2-b63c3d1e895f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:57.655094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ade6-cbdc-428e-95f2-b63c3d1e895f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:57.688987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7629ade6-cbdc-428e-95f2-b63c3d1e895f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:57.690171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7629ade6-cbdc-428e-95f2-b63c3d1e895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:57.715511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:57.722718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe496aa5-6ff3-4266-8121-656a54851c84 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:57.763744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30983301-9dd6-4b96-8ae3-8c702df1a009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:57.764510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30983301-9dd6-4b96-8ae3-8c702df1a009 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:57.764789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30983301-9dd6-4b96-8ae3-8c702df1a009 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:57.765068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30983301-9dd6-4b96-8ae3-8c702df1a009 None admin] PATH_INFO: `/v3/groups/84cfc0496b5a4ea1bac088c831734672` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:57.766566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30983301-9dd6-4b96-8ae3-8c702df1a009 None admin] RBAC: Authorizing `identity:get_group(group_id=84cfc0496b5a4ea1bac088c831734672)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:57.768051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30983301-9dd6-4b96-8ae3-8c702df1a009 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:57.774295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 203/423] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:14:57 2022] GET /identity/v3/groups/84cfc0496b5a4ea1bac088c831734672 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:57.776955 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30983301-9dd6-4b96-8ae3-8c702df1a009 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:57.839615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 221/424] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:57.937163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30983301-9dd6-4b96-8ae3-8c702df1a009 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:57.989848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1502486-7a73-42ef-9f72-103028576223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:57.990457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1502486-7a73-42ef-9f72-103028576223 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:57.990733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1502486-7a73-42ef-9f72-103028576223 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:57.991008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1502486-7a73-42ef-9f72-103028576223 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:57.991760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1502486-7a73-42ef-9f72-103028576223 None admin] RBAC: Adding query filter params (group.id=84cfc0496b5a4ea1bac088c831734672, role.id=ebbe7434d1ec4bc4ba71737f55987dd9, scope.project.id=7849898da2494c4897ab5e0e09c9fe9e) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:57.992064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1502486-7a73-42ef-9f72-103028576223 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:57.993540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1502486-7a73-42ef-9f72-103028576223 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:58.007403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 204/425] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Sep 12 23:14:57 2022] GET /identity/v3/role_assignments?group.id=84cfc0496b5a4ea1bac088c831734672&scope.project.id=7849898da2494c4897ab5e0e09c9fe9e&role.id=ebbe7434d1ec4bc4ba71737f55987dd9 => generated 611 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:58.163229 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629ade6-cbdc-428e-95f2-b63c3d1e895f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:58.305662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:58.312253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629ade6-cbdc-428e-95f2-b63c3d1e895f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:58.353158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d73fb-5e66-47ba-bb0e-4fabce73c228 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:58.354097 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d73fb-5e66-47ba-bb0e-4fabce73c228 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:58.354572 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d73fb-5e66-47ba-bb0e-4fabce73c228 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:58.356176 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 205/426] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:58.361816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076e8e9-aa0e-4391-930f-65cde6d30428 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:58.362366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076e8e9-aa0e-4391-930f-65cde6d30428 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:58.362847 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076e8e9-aa0e-4391-930f-65cde6d30428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:58.371296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbb7f3-de00-446f-8222-1b1d45c8c2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:58.371296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbb7f3-de00-446f-8222-1b1d45c8c2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:58.371528 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbb7f3-de00-446f-8222-1b1d45c8c2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:58.371789 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbb7f3-de00-446f-8222-1b1d45c8c2d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:58.385697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbb7f3-de00-446f-8222-1b1d45c8c2d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:58.386655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-2076e8e9-aa0e-4391-930f-65cde6d30428 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:58.388094 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2076e8e9-aa0e-4391-930f-65cde6d30428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-1ebbb7f3-de00-446f-8222-1b1d45c8c2d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:58.390573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:58.391508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:58.392355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:58.392355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:58.392355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:58.392355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:58.392355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:58.392355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:58.392355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:58.392355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:58.392355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:58.392355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:58.392355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:14:58.392355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:58.393491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 222/427] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:58.398713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbb7f3-de00-446f-8222-1b1d45c8c2d8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:58.416541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:58.476793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 206/428] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:58.532500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbb7f3-de00-446f-8222-1b1d45c8c2d8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:58.574431 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f500e9-4b63-4135-979b-988275ff4271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:58.575315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f500e9-4b63-4135-979b-988275ff4271 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:58.575795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f500e9-4b63-4135-979b-988275ff4271 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:58.576243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f500e9-4b63-4135-979b-988275ff4271 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:58.577102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f500e9-4b63-4135-979b-988275ff4271 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:58.577581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f500e9-4b63-4135-979b-988275ff4271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:58.579138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f500e9-4b63-4135-979b-988275ff4271 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:58.589211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 223/429] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:14:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:58.595136 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076e8e9-aa0e-4391-930f-65cde6d30428 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:58.705005 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076e8e9-aa0e-4391-930f-65cde6d30428 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:58.741482 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873c3e3-6377-459b-9b36-840e9099f3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:58.742387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873c3e3-6377-459b-9b36-840e9099f3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:58.742868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873c3e3-6377-459b-9b36-840e9099f3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:58.743912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873c3e3-6377-459b-9b36-840e9099f3c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:58.771802 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:58.824129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873c3e3-6377-459b-9b36-840e9099f3c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:58.825950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873c3e3-6377-459b-9b36-840e9099f3c7 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:58.850897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f500e9-4b63-4135-979b-988275ff4271 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:58.874160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:58.944353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 207/430] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:58.952851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873c3e3-6377-459b-9b36-840e9099f3c7 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:58.974210 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f500e9-4b63-4135-979b-988275ff4271 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:59.015257 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:59.016374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19532bc6-1bff-49ee-9ffb-a1bdac6ea2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:59.017316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19532bc6-1bff-49ee-9ffb-a1bdac6ea2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:59.017821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19532bc6-1bff-49ee-9ffb-a1bdac6ea2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:59.018293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19532bc6-1bff-49ee-9ffb-a1bdac6ea2a2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:59.033699 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19532bc6-1bff-49ee-9ffb-a1bdac6ea2a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-19532bc6-1bff-49ee-9ffb-a1bdac6ea2a2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:59.039474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:59.041148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:59.041963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:59.041963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:59.041963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:59.041963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:59.041963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:59.041963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:59.041963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:59.041963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:59.041963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:59.041963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:59.041963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:14:59.041963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:14:59.044636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 224/431] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:14:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:14:59.052340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19532bc6-1bff-49ee-9ffb-a1bdac6ea2a2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:59.076539 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873c3e3-6377-459b-9b36-840e9099f3c7 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:59.122805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc78d30-2a53-4aa0-be7f-5342a973ba8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:59.123829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc78d30-2a53-4aa0-be7f-5342a973ba8b None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:59.124565 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc78d30-2a53-4aa0-be7f-5342a973ba8b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:59.124999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc78d30-2a53-4aa0-be7f-5342a973ba8b None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:59.126297 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc78d30-2a53-4aa0-be7f-5342a973ba8b None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:59.128376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc78d30-2a53-4aa0-be7f-5342a973ba8b None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:59.130807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 208/432] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:58 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:59.137870 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc78d30-2a53-4aa0-be7f-5342a973ba8b None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:59.168764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:59.205561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19532bc6-1bff-49ee-9ffb-a1bdac6ea2a2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:59.249966 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd108b-2e4b-4c16-b1aa-8123c1c0de1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:59.250579 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd108b-2e4b-4c16-b1aa-8123c1c0de1c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:59.251201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd108b-2e4b-4c16-b1aa-8123c1c0de1c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:59.251657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd108b-2e4b-4c16-b1aa-8123c1c0de1c None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:59.253653 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfd108b-2e4b-4c16-b1aa-8123c1c0de1c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:59.254194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfd108b-2e4b-4c16-b1aa-8123c1c0de1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:59.256031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfd108b-2e4b-4c16-b1aa-8123c1c0de1c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:59.265626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 225/433] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:14:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:59.271761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c6178-555b-4ec4-b7c7-44cd6472497c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:59.272053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c6178-555b-4ec4-b7c7-44cd6472497c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:59.272328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c6178-555b-4ec4-b7c7-44cd6472497c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:59.273309 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc78d30-2a53-4aa0-be7f-5342a973ba8b None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:59.273818 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 226/434] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:59.275710 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c6178-555b-4ec4-b7c7-44cd6472497c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:59.308021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:59.321162 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e4d76-3935-4ea3-b842-569511a0e6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:59.321824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e4d76-3935-4ea3-b842-569511a0e6da None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:59.322104 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e4d76-3935-4ea3-b842-569511a0e6da None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:59.322376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e4d76-3935-4ea3-b842-569511a0e6da None admin] PATH_INFO: `/v3/groups/84cfc0496b5a4ea1bac088c831734672` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:59.323509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e4d76-3935-4ea3-b842-569511a0e6da None admin] RBAC: Authorizing `identity:get_group(group_id=84cfc0496b5a4ea1bac088c831734672)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:59.325419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e4d76-3935-4ea3-b842-569511a0e6da None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:59.332015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 209/435] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:14:59 2022] GET /identity/v3/groups/84cfc0496b5a4ea1bac088c831734672 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:59.333398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54535d-ba2b-4e1b-ad78-8863a181648e None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:59.333688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54535d-ba2b-4e1b-ad78-8863a181648e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:59.333991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54535d-ba2b-4e1b-ad78-8863a181648e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:59.352015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-5b54535d-ba2b-4e1b-ad78-8863a181648e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:59.353747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5b54535d-ba2b-4e1b-ad78-8863a181648e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:59.385889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c6178-555b-4ec4-b7c7-44cd6472497c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:59.419314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedec9c-6999-4af5-b084-ae10979136ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:59.419467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 210/436] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:14:59.419951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedec9c-6999-4af5-b084-ae10979136ce None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:59.420232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedec9c-6999-4af5-b084-ae10979136ce None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:59.420330 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54535d-ba2b-4e1b-ad78-8863a181648e None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:59.420676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedec9c-6999-4af5-b084-ae10979136ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:59.439939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:59.499700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedec9c-6999-4af5-b084-ae10979136ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:59.501143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedec9c-6999-4af5-b084-ae10979136ce None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:59.520239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54535d-ba2b-4e1b-ad78-8863a181648e None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:59.544451 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:59.559292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746026ee-3a1c-45ba-a161-0d7faef657c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:59.560136 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746026ee-3a1c-45ba-a161-0d7faef657c2 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:59.560419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746026ee-3a1c-45ba-a161-0d7faef657c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:59.560690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746026ee-3a1c-45ba-a161-0d7faef657c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:59.561406 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746026ee-3a1c-45ba-a161-0d7faef657c2 None admin] RBAC: Adding query filter params (group.id=84cfc0496b5a4ea1bac088c831734672, role.id=ebbe7434d1ec4bc4ba71737f55987dd9, scope.project.id=83919a658b6446d098540ce590417c23) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:14:59.561715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746026ee-3a1c-45ba-a161-0d7faef657c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:59.563250 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746026ee-3a1c-45ba-a161-0d7faef657c2 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:59.569647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 211/437] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Sep 12 23:14:59 2022] GET /identity/v3/role_assignments?group.id=84cfc0496b5a4ea1bac088c831734672&scope.project.id=83919a658b6446d098540ce590417c23&role.id=ebbe7434d1ec4bc4ba71737f55987dd9 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:59.570552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dac113-585c-425e-9b4c-822ccfbfc7e6 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:59.570831 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dac113-585c-425e-9b4c-822ccfbfc7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:59.571099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dac113-585c-425e-9b4c-822ccfbfc7e6 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:59.573140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 212/438] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:14:59.578738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d15c0-a890-4d24-bd45-eef1a92d4154 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:59.579013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d15c0-a890-4d24-bd45-eef1a92d4154 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:59.579281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d15c0-a890-4d24-bd45-eef1a92d4154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:59.598637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-ff8d15c0-a890-4d24-bd45-eef1a92d4154 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:14:59.599811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ff8d15c0-a890-4d24-bd45-eef1a92d4154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:14:59.624442 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 227/439] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:14:59.629710 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedec9c-6999-4af5-b084-ae10979136ce None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:59.681143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:59.703434 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 213/440] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:14:59.705486 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d15c0-a890-4d24-bd45-eef1a92d4154 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:59.747666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedec9c-6999-4af5-b084-ae10979136ce None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:59.799235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290dc0d-d728-467d-bb35-b1e26b2dc77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:59.800206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290dc0d-d728-467d-bb35-b1e26b2dc77c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:59.800697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290dc0d-d728-467d-bb35-b1e26b2dc77c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:59.801190 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290dc0d-d728-467d-bb35-b1e26b2dc77c None admin] PATH_INFO: `/v3/projects/7849898da2494c4897ab5e0e09c9fe9e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:59.802536 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290dc0d-d728-467d-bb35-b1e26b2dc77c None admin] RBAC: Authorizing `identity:get_project(project_id=7849898da2494c4897ab5e0e09c9fe9e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:59.804418 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290dc0d-d728-467d-bb35-b1e26b2dc77c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:59.806248 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 228/441] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:14:59 2022] GET /identity/v3/projects/7849898da2494c4897ab5e0e09c9fe9e => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:14:59.811731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290dc0d-d728-467d-bb35-b1e26b2dc77c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:59.828114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d15c0-a890-4d24-bd45-eef1a92d4154 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:59.845435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:59.867364 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba2d51-6c66-4df7-99a1-bbb8d99ce244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:59.867967 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba2d51-6c66-4df7-99a1-bbb8d99ce244 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:59.868789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba2d51-6c66-4df7-99a1-bbb8d99ce244 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:59.869063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba2d51-6c66-4df7-99a1-bbb8d99ce244 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:59.880305 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba2d51-6c66-4df7-99a1-bbb8d99ce244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-77ba2d51-6c66-4df7-99a1-bbb8d99ce244 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:14:59.885247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:14:59.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:14:59.886922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:14:59.886922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:14:59.886922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:14:59.886922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:14:59.886922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:14:59.886922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:14:59.886922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:14:59.886922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:14:59.886922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:14:59.886922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:14:59.886922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:14:59.886922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:14:59.886922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 214/442] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:14:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:14:59.892133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba2d51-6c66-4df7-99a1-bbb8d99ce244 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:14:59.939783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290dc0d-d728-467d-bb35-b1e26b2dc77c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:14:59.949971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:14:59.979279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd51825-1ff2-4551-a7c2-cdf5442fc7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:14:59.979937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd51825-1ff2-4551-a7c2-cdf5442fc7ce None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:14:59.980219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd51825-1ff2-4551-a7c2-cdf5442fc7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:14:59.980665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd51825-1ff2-4551-a7c2-cdf5442fc7ce None admin] PATH_INFO: `/v3/groups/84cfc0496b5a4ea1bac088c831734672` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:14:59.982294 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd51825-1ff2-4551-a7c2-cdf5442fc7ce None admin] RBAC: Authorizing `identity:get_group(group_id=84cfc0496b5a4ea1bac088c831734672)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:14:59.983412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd51825-1ff2-4551-a7c2-cdf5442fc7ce None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:14:59.985031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 229/443] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:14:59 2022] GET /identity/v3/groups/84cfc0496b5a4ea1bac088c831734672 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:14:59.985791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd51825-1ff2-4551-a7c2-cdf5442fc7ce None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:00.006829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba2d51-6c66-4df7-99a1-bbb8d99ce244 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:00.041499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcf08a-a3ed-4476-b814-880192005684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:00.042097 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcf08a-a3ed-4476-b814-880192005684 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:00.042378 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcf08a-a3ed-4476-b814-880192005684 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:00.042650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcf08a-a3ed-4476-b814-880192005684 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:00.043306 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fcf08a-a3ed-4476-b814-880192005684 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:00.043637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fcf08a-a3ed-4476-b814-880192005684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:00.045002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fcf08a-a3ed-4476-b814-880192005684 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:00.053064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 215/444] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:14:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:00.053870 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcf08a-a3ed-4476-b814-880192005684 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:00.087335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd51825-1ff2-4551-a7c2-cdf5442fc7ce None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:00.093760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:00.121515 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5631e7-0b5b-4fdf-8b9d-f22272714381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:00.122070 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5631e7-0b5b-4fdf-8b9d-f22272714381 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:00.122352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5631e7-0b5b-4fdf-8b9d-f22272714381 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:00.122618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5631e7-0b5b-4fdf-8b9d-f22272714381 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:00.132378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5631e7-0b5b-4fdf-8b9d-f22272714381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ff5631e7-0b5b-4fdf-8b9d-f22272714381 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:00.137763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:00.138632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:00.139420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:00.139420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:00.139420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:00.139420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:00.139420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:00.139420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:00.139420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:00.139420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:00.139420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:00.139420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:00.139420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:15:00.139420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:00.139420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 230/445] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:14:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:00.140104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5631e7-0b5b-4fdf-8b9d-f22272714381 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:00.154829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcf08a-a3ed-4476-b814-880192005684 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:00.176285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:00.188590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bff97-03e0-4513-92d7-176b0d72a8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:00.189190 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bff97-03e0-4513-92d7-176b0d72a8cd None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:00.189466 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bff97-03e0-4513-92d7-176b0d72a8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:00.189738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bff97-03e0-4513-92d7-176b0d72a8cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:00.190452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bff97-03e0-4513-92d7-176b0d72a8cd None admin] RBAC: Adding query filter params (group.id=84cfc0496b5a4ea1bac088c831734672, role.id=31058b9633b44d7ca64f103119987fd9, scope.project.id=7849898da2494c4897ab5e0e09c9fe9e) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:00.190758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bff97-03e0-4513-92d7-176b0d72a8cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:00.192227 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bff97-03e0-4513-92d7-176b0d72a8cd None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:00.197751 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 216/446] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Sep 12 23:15:00 2022] GET /identity/v3/role_assignments?group.id=84cfc0496b5a4ea1bac088c831734672&scope.project.id=7849898da2494c4897ab5e0e09c9fe9e&role.id=31058b9633b44d7ca64f103119987fd9 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:00.198516 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bff97-03e0-4513-92d7-176b0d72a8cd None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:00.263948 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5631e7-0b5b-4fdf-8b9d-f22272714381 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:00.302529 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c8216-4b7a-4fab-b678-55ed8e287079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:00.303194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c8216-4b7a-4fab-b678-55ed8e287079 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:00.303543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c8216-4b7a-4fab-b678-55ed8e287079 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:00.303887 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c8216-4b7a-4fab-b678-55ed8e287079 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:00.311055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bff97-03e0-4513-92d7-176b0d72a8cd None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:00.312973 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:00.350886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600ace5-4909-4c0e-837b-f4cf9f087882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:00.351518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600ace5-4909-4c0e-837b-f4cf9f087882 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:00.351829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600ace5-4909-4c0e-837b-f4cf9f087882 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:00.352105 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600ace5-4909-4c0e-837b-f4cf9f087882 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:00.353405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600ace5-4909-4c0e-837b-f4cf9f087882 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:00.353713 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600ace5-4909-4c0e-837b-f4cf9f087882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:00.355138 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600ace5-4909-4c0e-837b-f4cf9f087882 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:00.364599 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 217/447] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:00.369995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600ace5-4909-4c0e-837b-f4cf9f087882 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:00.391312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c8216-4b7a-4fab-b678-55ed8e287079 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:00.391900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c8216-4b7a-4fab-b678-55ed8e287079 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:00.435125 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:00.487400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600ace5-4909-4c0e-837b-f4cf9f087882 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:00.527541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f67492-e0ff-4038-9743-042e3f7406a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:00.528210 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f67492-e0ff-4038-9743-042e3f7406a2 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:00.528495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f67492-e0ff-4038-9743-042e3f7406a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:00.528774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f67492-e0ff-4038-9743-042e3f7406a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:00.529837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 231/448] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:15:00.535818 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c8216-4b7a-4fab-b678-55ed8e287079 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:00.568805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:00.615399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f67492-e0ff-4038-9743-042e3f7406a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:00.616903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f67492-e0ff-4038-9743-042e3f7406a2 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:00.655109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c8216-4b7a-4fab-b678-55ed8e287079 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:00.702055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc21e51-58b8-48dc-a4c4-3f255809de0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:00.702692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc21e51-58b8-48dc-a4c4-3f255809de0e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:00.702969 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc21e51-58b8-48dc-a4c4-3f255809de0e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:00.703700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc21e51-58b8-48dc-a4c4-3f255809de0e None admin] PATH_INFO: `/v3/projects/5b515d316021473bab5135bffb78aaff` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:00.705464 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc21e51-58b8-48dc-a4c4-3f255809de0e None admin] RBAC: Authorizing `identity:get_project(project_id=5b515d316021473bab5135bffb78aaff)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:00.706282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc21e51-58b8-48dc-a4c4-3f255809de0e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:00.708162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 232/449] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:15:00 2022] GET /identity/v3/projects/5b515d316021473bab5135bffb78aaff => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:00.713518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc21e51-58b8-48dc-a4c4-3f255809de0e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:00.718396 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:00.735501 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 218/450] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:15:00.741075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f67492-e0ff-4038-9743-042e3f7406a2 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:00.800055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:00.820363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc21e51-58b8-48dc-a4c4-3f255809de0e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:00.837725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f67492-e0ff-4038-9743-042e3f7406a2 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:00.852973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403dc8d-9d24-4d9d-af56-c7f96c0a7498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:00.853603 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403dc8d-9d24-4d9d-af56-c7f96c0a7498 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:00.853887 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403dc8d-9d24-4d9d-af56-c7f96c0a7498 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:00.854160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403dc8d-9d24-4d9d-af56-c7f96c0a7498 None admin] PATH_INFO: `/v3/groups/9c7caa2d948e42c1ae1bc416139e40ed` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:00.855300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3403dc8d-9d24-4d9d-af56-c7f96c0a7498 None admin] RBAC: Authorizing `identity:get_group(group_id=9c7caa2d948e42c1ae1bc416139e40ed)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:00.856795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3403dc8d-9d24-4d9d-af56-c7f96c0a7498 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:00.858365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 233/451] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:15:00 2022] GET /identity/v3/groups/9c7caa2d948e42c1ae1bc416139e40ed => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:00.864051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403dc8d-9d24-4d9d-af56-c7f96c0a7498 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:00.872170 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af077f-e42c-4ad8-b7c2-253ec5b39885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:00.872806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af077f-e42c-4ad8-b7c2-253ec5b39885 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:00.873517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af077f-e42c-4ad8-b7c2-253ec5b39885 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:00.873821 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af077f-e42c-4ad8-b7c2-253ec5b39885 None admin] PATH_INFO: `/v3/projects/83919a658b6446d098540ce590417c23` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:00.875359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af077f-e42c-4ad8-b7c2-253ec5b39885 None admin] RBAC: Authorizing `identity:get_project(project_id=83919a658b6446d098540ce590417c23)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:00.877173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af077f-e42c-4ad8-b7c2-253ec5b39885 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:00.879075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 219/452] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:15:00 2022] GET /identity/v3/projects/83919a658b6446d098540ce590417c23 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:00.884235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af077f-e42c-4ad8-b7c2-253ec5b39885 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:00.937081 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:00.968056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403dc8d-9d24-4d9d-af56-c7f96c0a7498 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:00.981417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af077f-e42c-4ad8-b7c2-253ec5b39885 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:01.000959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e331ff3-9bef-45f8-8a6f-8c0cd55f8970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:01.001560 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e331ff3-9bef-45f8-8a6f-8c0cd55f8970 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:01.001840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e331ff3-9bef-45f8-8a6f-8c0cd55f8970 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:01.002115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e331ff3-9bef-45f8-8a6f-8c0cd55f8970 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:01.002839 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e331ff3-9bef-45f8-8a6f-8c0cd55f8970 None admin] RBAC: Adding query filter params (group.id=9c7caa2d948e42c1ae1bc416139e40ed, role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=5b515d316021473bab5135bffb78aaff) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:01.003152 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e331ff3-9bef-45f8-8a6f-8c0cd55f8970 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:01.004601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e331ff3-9bef-45f8-8a6f-8c0cd55f8970 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:01.009037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:01.010249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 234/453] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Sep 12 23:15:00 2022] GET /identity/v3/role_assignments?group.id=9c7caa2d948e42c1ae1bc416139e40ed&scope.project.id=5b515d316021473bab5135bffb78aaff&role.id=23538dc4c770454e80dd11ed8e498539 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:01.014256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060123c-d3b3-4e27-91f5-d22ba6d07352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:01.014900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060123c-d3b3-4e27-91f5-d22ba6d07352 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:01.019901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060123c-d3b3-4e27-91f5-d22ba6d07352 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:01.019901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060123c-d3b3-4e27-91f5-d22ba6d07352 None admin] PATH_INFO: `/v3/groups/84cfc0496b5a4ea1bac088c831734672` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:01.019901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060123c-d3b3-4e27-91f5-d22ba6d07352 None admin] RBAC: Authorizing `identity:get_group(group_id=84cfc0496b5a4ea1bac088c831734672)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:01.019901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060123c-d3b3-4e27-91f5-d22ba6d07352 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:01.020337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 220/454] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Sep 12 23:15:00 2022] GET /identity/v3/groups/84cfc0496b5a4ea1bac088c831734672 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:01.026589 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e331ff3-9bef-45f8-8a6f-8c0cd55f8970 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:01.130678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e331ff3-9bef-45f8-8a6f-8c0cd55f8970 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:01.166137 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e170cf1-a16c-42ef-848d-626424884f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:01.166741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e170cf1-a16c-42ef-848d-626424884f1e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:01.167016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e170cf1-a16c-42ef-848d-626424884f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:01.167283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e170cf1-a16c-42ef-848d-626424884f1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:01.168033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e170cf1-a16c-42ef-848d-626424884f1e None admin] RBAC: Adding query filter params (group.id=84cfc0496b5a4ea1bac088c831734672, role.id=31058b9633b44d7ca64f103119987fd9, scope.project.id=83919a658b6446d098540ce590417c23) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:01.168339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e170cf1-a16c-42ef-848d-626424884f1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:01.169836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e170cf1-a16c-42ef-848d-626424884f1e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:01.172756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:01.176761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 235/455] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Sep 12 23:15:01 2022] GET /identity/v3/role_assignments?group.id=84cfc0496b5a4ea1bac088c831734672&scope.project.id=83919a658b6446d098540ce590417c23&role.id=31058b9633b44d7ca64f103119987fd9 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:03.782671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb93d6-5200-4246-a73e-00e802f0dba5 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:03.783442 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb93d6-5200-4246-a73e-00e802f0dba5 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:03.783519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb93d6-5200-4246-a73e-00e802f0dba5 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:03.785133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 221/456] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:03.791188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9249c0-39f1-453b-8610-d2d80decb09d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:03.791635 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9249c0-39f1-453b-8610-d2d80decb09d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:03.791811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9249c0-39f1-453b-8610-d2d80decb09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:03.820583 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0d9249c0-39f1-453b-8610-d2d80decb09d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:03.822336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0d9249c0-39f1-453b-8610-d2d80decb09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:03.930250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 236/457] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:15:04.102342 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9377f-3b9f-457b-b69d-3fcf61d30339 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:04.102542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9377f-3b9f-457b-b69d-3fcf61d30339 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:04.102797 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9377f-3b9f-457b-b69d-3fcf61d30339 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:04.104169 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 222/458] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:04.109372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350f544-f1f7-40b4-b78e-e294940a666b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:04.109659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350f544-f1f7-40b4-b78e-e294940a666b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:04.109933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350f544-f1f7-40b4-b78e-e294940a666b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:04.136561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0350f544-f1f7-40b4-b78e-e294940a666b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:04.137762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0350f544-f1f7-40b4-b78e-e294940a666b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:04.223773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 237/459] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:04.349502 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9377f-3b9f-457b-b69d-3fcf61d30339 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:04.436684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48d6a6-fb04-49d9-8e2e-e5904cb6e1ba None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:04.437006 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48d6a6-fb04-49d9-8e2e-e5904cb6e1ba None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:04.437243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48d6a6-fb04-49d9-8e2e-e5904cb6e1ba None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:04.438838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 238/460] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:04.447393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fde6b2-9bd9-43be-9524-88073c47d2bc None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:04.447393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fde6b2-9bd9-43be-9524-88073c47d2bc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:04.447393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fde6b2-9bd9-43be-9524-88073c47d2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:04.474740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-37fde6b2-9bd9-43be-9524-88073c47d2bc None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:04.476003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-37fde6b2-9bd9-43be-9524-88073c47d2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:04.491647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9377f-3b9f-457b-b69d-3fcf61d30339 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:04.541131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187f8c3-92b6-45c7-853e-a4cba7e7a09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:04.541341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:04.542395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187f8c3-92b6-45c7-853e-a4cba7e7a09f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:04.543422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187f8c3-92b6-45c7-853e-a4cba7e7a09f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:04.543900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187f8c3-92b6-45c7-853e-a4cba7e7a09f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:04.556810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187f8c3-92b6-45c7-853e-a4cba7e7a09f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:04.558511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 239/461] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:15:04.558834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187f8c3-92b6-45c7-853e-a4cba7e7a09f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:04.560376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b79d5-d5e3-4396-a167-8fa23e5d2be1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:04.560376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b79d5-d5e3-4396-a167-8fa23e5d2be1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:04.560614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b79d5-d5e3-4396-a167-8fa23e5d2be1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:04.566096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 240/462] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:04.567375 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72228612-7e69-402c-879f-38db992cf3b7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:04.567496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72228612-7e69-402c-879f-38db992cf3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:04.567835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72228612-7e69-402c-879f-38db992cf3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-8187f8c3-92b6-45c7-853e-a4cba7e7a09f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:04.592728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:04.593681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:04.595512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:04.595512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:04.595512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:04.595512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 223/463] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:04.604608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187f8c3-92b6-45c7-853e-a4cba7e7a09f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:04.604891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-72228612-7e69-402c-879f-38db992cf3b7 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:04.606105 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-72228612-7e69-402c-879f-38db992cf3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:04.667286 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:04.691120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 241/464] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:04.692034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72228612-7e69-402c-879f-38db992cf3b7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:04.719636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187f8c3-92b6-45c7-853e-a4cba7e7a09f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:04.757129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7435706-bdba-404f-8699-758a0fb75e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:04.758024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7435706-bdba-404f-8699-758a0fb75e6e None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:04.758311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7435706-bdba-404f-8699-758a0fb75e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:04.758646 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7435706-bdba-404f-8699-758a0fb75e6e None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:04.759293 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7435706-bdba-404f-8699-758a0fb75e6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:04.759621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7435706-bdba-404f-8699-758a0fb75e6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:04.761874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7435706-bdba-404f-8699-758a0fb75e6e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:04.788004 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 224/465] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:04.794560 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7435706-bdba-404f-8699-758a0fb75e6e None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:04.831881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72228612-7e69-402c-879f-38db992cf3b7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:04.854888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:04.873432 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f88d3-4c57-4b58-8c13-24f0a6b8ac0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:04.874542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f88d3-4c57-4b58-8c13-24f0a6b8ac0d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:04.874963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f88d3-4c57-4b58-8c13-24f0a6b8ac0d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:04.875405 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f88d3-4c57-4b58-8c13-24f0a6b8ac0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:04.887929 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f88d3-4c57-4b58-8c13-24f0a6b8ac0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:04.889368 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f88d3-4c57-4b58-8c13-24f0a6b8ac0d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-cc9f88d3-4c57-4b58-8c13-24f0a6b8ac0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:04.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:04.906791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:04.907666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:04.907666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:04.907666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:04.907666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 242/466] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:04.908165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f88d3-4c57-4b58-8c13-24f0a6b8ac0d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:04.943489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7435706-bdba-404f-8699-758a0fb75e6e None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:04.967594 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:04.994507 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979a6e8-88f3-49a2-9c28-df7d654328fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:04.996086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979a6e8-88f3-49a2-9c28-df7d654328fc None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:04.996368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979a6e8-88f3-49a2-9c28-df7d654328fc None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:04.996640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979a6e8-88f3-49a2-9c28-df7d654328fc None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:04.997299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979a6e8-88f3-49a2-9c28-df7d654328fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:04.998757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979a6e8-88f3-49a2-9c28-df7d654328fc None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:05.021334 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 225/467] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:15:04 2022] POST /identity/v3/users => generated 246 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:15:05.022159 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979a6e8-88f3-49a2-9c28-df7d654328fc None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:05.034349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f88d3-4c57-4b58-8c13-24f0a6b8ac0d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:05.078240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5af2e-a543-4078-96dc-c0a037d7bb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:05.079295 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5af2e-a543-4078-96dc-c0a037d7bb1b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:05.079856 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5af2e-a543-4078-96dc-c0a037d7bb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:05.080769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5af2e-a543-4078-96dc-c0a037d7bb1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:05.095290 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5af2e-a543-4078-96dc-c0a037d7bb1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:05.097580 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5af2e-a543-4078-96dc-c0a037d7bb1b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-54e5af2e-a543-4078-96dc-c0a037d7bb1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:05.116985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:05.118013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:05.118873 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:05.118873 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:05.118873 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:05.121801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 243/468] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:05.123250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5af2e-a543-4078-96dc-c0a037d7bb1b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:05.153409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979a6e8-88f3-49a2-9c28-df7d654328fc None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:05.162919 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:05.202117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc5b07-524a-484c-a806-a45b1741aa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:05.203099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc5b07-524a-484c-a806-a45b1741aa5b None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:05.203481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc5b07-524a-484c-a806-a45b1741aa5b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:05.203885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc5b07-524a-484c-a806-a45b1741aa5b None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:05.204907 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc5b07-524a-484c-a806-a45b1741aa5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:05.205677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc5b07-524a-484c-a806-a45b1741aa5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:05.207480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc5b07-524a-484c-a806-a45b1741aa5b None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:05.223411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 226/469] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:05.224555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc5b07-524a-484c-a806-a45b1741aa5b None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:05.252265 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5af2e-a543-4078-96dc-c0a037d7bb1b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:05.260637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:05.298842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9cc577-3b09-4b32-b165-902fa4ac317f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:05.299482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cc577-3b09-4b32-b165-902fa4ac317f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:05.299797 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cc577-3b09-4b32-b165-902fa4ac317f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:05.300073 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cc577-3b09-4b32-b165-902fa4ac317f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:05.314457 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9cc577-3b09-4b32-b165-902fa4ac317f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:05.316821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9cc577-3b09-4b32-b165-902fa4ac317f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4a9cc577-3b09-4b32-b165-902fa4ac317f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:05.335321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:05.336306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:05.337200 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:05.337200 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:05.337200 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:05.338618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 244/470] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:05.339519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9cc577-3b09-4b32-b165-902fa4ac317f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:05.366347 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc5b07-524a-484c-a806-a45b1741aa5b None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:05.411118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fca4eb-3edb-4978-829f-a2405ffd21dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:05.411891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fca4eb-3edb-4978-829f-a2405ffd21dc None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:05.412177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fca4eb-3edb-4978-829f-a2405ffd21dc None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:05.412452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fca4eb-3edb-4978-829f-a2405ffd21dc None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:05.413512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fca4eb-3edb-4978-829f-a2405ffd21dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:05.413835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fca4eb-3edb-4978-829f-a2405ffd21dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:05.415724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fca4eb-3edb-4978-829f-a2405ffd21dc None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:05.430238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 227/471] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:05.431908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fca4eb-3edb-4978-829f-a2405ffd21dc None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:05.433673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:05.468537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9cc577-3b09-4b32-b165-902fa4ac317f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:05.509687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c634c6-f4f9-40ed-be67-a5e34ac94864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:05.511199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c634c6-f4f9-40ed-be67-a5e34ac94864 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:05.511485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c634c6-f4f9-40ed-be67-a5e34ac94864 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:05.511789 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c634c6-f4f9-40ed-be67-a5e34ac94864 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:05.512460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c634c6-f4f9-40ed-be67-a5e34ac94864 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:05.514022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c634c6-f4f9-40ed-be67-a5e34ac94864 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:05.531398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:05.542591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 245/472] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:15:05 2022] POST /identity/v3/users => generated 248 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:15:05.543697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c634c6-f4f9-40ed-be67-a5e34ac94864 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:05.549259 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fca4eb-3edb-4978-829f-a2405ffd21dc None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:05.593729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39baf392-0e97-4fb0-8c68-7a8e4d94a322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:05.594518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baf392-0e97-4fb0-8c68-7a8e4d94a322 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:05.594900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baf392-0e97-4fb0-8c68-7a8e4d94a322 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:05.595272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baf392-0e97-4fb0-8c68-7a8e4d94a322 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:05.596104 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39baf392-0e97-4fb0-8c68-7a8e4d94a322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:05.596523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39baf392-0e97-4fb0-8c68-7a8e4d94a322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:05.598576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39baf392-0e97-4fb0-8c68-7a8e4d94a322 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:05.617152 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 228/473] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:05.622714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39baf392-0e97-4fb0-8c68-7a8e4d94a322 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:05.734164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c634c6-f4f9-40ed-be67-a5e34ac94864 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:05.787484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:05.788677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39baf392-0e97-4fb0-8c68-7a8e4d94a322 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:05.798547 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d43c5a-043f-453a-956c-2ac7f13f69b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:05.799492 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d43c5a-043f-453a-956c-2ac7f13f69b1 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:05.799987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d43c5a-043f-453a-956c-2ac7f13f69b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:05.800269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d43c5a-043f-453a-956c-2ac7f13f69b1 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:05.801072 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d43c5a-043f-453a-956c-2ac7f13f69b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:05.802479 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d43c5a-043f-453a-956c-2ac7f13f69b1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:05.829289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 246/474] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:15:05 2022] POST /identity/v3/users => generated 249 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:15:05.830529 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cec3a6-87e6-4241-9355-43ce96625066 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:05.830814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cec3a6-87e6-4241-9355-43ce96625066 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:05.831091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cec3a6-87e6-4241-9355-43ce96625066 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:05.832354 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87967d-cc5d-4b35-8bae-2bbb470cdf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:05.832581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 247/475] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:05.833013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87967d-cc5d-4b35-8bae-2bbb470cdf88 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:05.833248 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87967d-cc5d-4b35-8bae-2bbb470cdf88 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:05.833590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87967d-cc5d-4b35-8bae-2bbb470cdf88 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:05.834363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87967d-cc5d-4b35-8bae-2bbb470cdf88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:05.836764 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87967d-cc5d-4b35-8bae-2bbb470cdf88 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:05.845534 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397d9b1-0898-4a08-8b1a-6920d42caa31 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:05.846181 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397d9b1-0898-4a08-8b1a-6920d42caa31 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:05.846670 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397d9b1-0898-4a08-8b1a-6920d42caa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:05.865483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 229/476] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:15:05 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:15:05.873555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1397d9b1-0898-4a08-8b1a-6920d42caa31 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:05.875168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1397d9b1-0898-4a08-8b1a-6920d42caa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:05.963778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 248/477] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:15:06.240773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87967d-cc5d-4b35-8bae-2bbb470cdf88 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:06.245518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:06.361198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87967d-cc5d-4b35-8bae-2bbb470cdf88 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:06.397966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060cffa-9263-4b0c-940a-3a44e8036749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:06.398970 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060cffa-9263-4b0c-940a-3a44e8036749 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:06.400001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060cffa-9263-4b0c-940a-3a44e8036749 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:06.400459 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060cffa-9263-4b0c-940a-3a44e8036749 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:06.401337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060cffa-9263-4b0c-940a-3a44e8036749 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:06.402915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060cffa-9263-4b0c-940a-3a44e8036749 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:06.416610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.sql.core [None req-8060cffa-9263-4b0c-940a-3a44e8036749 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 23:15:06.416610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 12 23:15:06.416610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [parameters: {'id': '5d156315a8374934b0905b33b98c0875', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 12 23:15:06.416610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81416) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-8060cffa-9263-4b0c-940a-3a44e8036749 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application conn.query(q) Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application result.read() Sep 12 23:15:06.433663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return ref.to_dict() Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 23:15:06.434557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application next(self.gen) Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application yield resource Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application next(self.gen) Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise self.value Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application session.commit() Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 23:15:06.435457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.session.flush() Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application compat.raise_( Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 23:15:06.436808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 23:15:06.437660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application conn.query(q) Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application result.read() Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 23:15:06.438458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [parameters: {'id': '5d156315a8374934b0905b33b98c0875', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:06.439228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 12 23:15:06.440077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:06.443195 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 230/478] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Sep 12 23:15:06 2022] POST /identity/v3/roles => generated 144 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:06.449255 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397d9b1-0898-4a08-8b1a-6920d42caa31 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:06.599020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397d9b1-0898-4a08-8b1a-6920d42caa31 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:06.621936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:06.644980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1808bb4-8f06-49aa-9f71-d4736d3d7142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:06.645615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1808bb4-8f06-49aa-9f71-d4736d3d7142 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:06.645892 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1808bb4-8f06-49aa-9f71-d4736d3d7142 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:06.646163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1808bb4-8f06-49aa-9f71-d4736d3d7142 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:06.662138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1808bb4-8f06-49aa-9f71-d4736d3d7142 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f1808bb4-8f06-49aa-9f71-d4736d3d7142 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:06.667025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:06.667978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:06.668788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:06.668788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:06.668788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:06.668788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:06.668788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:06.668788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:06.668788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:06.668788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:06.668788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:06.668788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:06.668788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:15:06.668788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:06.668788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 249/479] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:15:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:06.676848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060cffa-9263-4b0c-940a-3a44e8036749 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:06.767786 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:06.810974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060cffa-9263-4b0c-940a-3a44e8036749 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:06.853909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a99ad-5cf1-4510-a58f-36edaa9c9b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:06.854754 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a99ad-5cf1-4510-a58f-36edaa9c9b6c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:06.855172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a99ad-5cf1-4510-a58f-36edaa9c9b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:06.855600 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a99ad-5cf1-4510-a58f-36edaa9c9b6c None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:06.856391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a99ad-5cf1-4510-a58f-36edaa9c9b6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:06.856846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a99ad-5cf1-4510-a58f-36edaa9c9b6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:06.858346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a99ad-5cf1-4510-a58f-36edaa9c9b6c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:06.866712 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 231/480] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:07.539635 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a4fbe-46e3-4377-980d-3214cf379f0a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:07.540175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a4fbe-46e3-4377-980d-3214cf379f0a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:07.540284 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a4fbe-46e3-4377-980d-3214cf379f0a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:07.542400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 250/481] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:07.547740 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98784684-b978-458c-991e-710040954b8e None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:07.548036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98784684-b978-458c-991e-710040954b8e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:07.548316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98784684-b978-458c-991e-710040954b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:07.567452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-98784684-b978-458c-991e-710040954b8e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:07.568703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-98784684-b978-458c-991e-710040954b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:07.638796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 232/482] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:07.934872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a4fbe-46e3-4377-980d-3214cf379f0a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:08.042263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a4fbe-46e3-4377-980d-3214cf379f0a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:08.077842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d36ae7-57bb-47fb-bca6-03bba94516d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:08.078816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d36ae7-57bb-47fb-bca6-03bba94516d8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:08.079299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d36ae7-57bb-47fb-bca6-03bba94516d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:08.079839 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d36ae7-57bb-47fb-bca6-03bba94516d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:08.091101 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d36ae7-57bb-47fb-bca6-03bba94516d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c0d36ae7-57bb-47fb-bca6-03bba94516d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:08.096309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:08.097207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:08.098014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:08.098014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:08.098014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:08.098014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:08.098014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:08.098014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:08.098014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:08.098014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:08.098014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:08.098014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:08.098014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:08.098014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:08.100654 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 251/483] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:08.107210 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98784684-b978-458c-991e-710040954b8e None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:08.187886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:08.233898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98784684-b978-458c-991e-710040954b8e None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:08.270367 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a4ac9-2868-4cb6-9a73-ad2e5e1ab76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:08.270980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a4ac9-2868-4cb6-9a73-ad2e5e1ab76f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:08.271264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a4ac9-2868-4cb6-9a73-ad2e5e1ab76f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:08.271616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a4ac9-2868-4cb6-9a73-ad2e5e1ab76f None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:08.272290 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a4ac9-2868-4cb6-9a73-ad2e5e1ab76f None admin] RBAC: Adding query filter params (name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:08.272597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a4ac9-2868-4cb6-9a73-ad2e5e1ab76f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:08.274130 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a4ac9-2868-4cb6-9a73-ad2e5e1ab76f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:08.286346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 233/484] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:08.292519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d36ae7-57bb-47fb-bca6-03bba94516d8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:08.325157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68baaf-5cea-4966-945b-44f68178ba38 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:08.325773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68baaf-5cea-4966-945b-44f68178ba38 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:08.326284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68baaf-5cea-4966-945b-44f68178ba38 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:08.328121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 234/485] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:08.333835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece46c9-d29e-42d2-8221-2fdd9db7f7d1 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:08.334527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece46c9-d29e-42d2-8221-2fdd9db7f7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:08.335338 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece46c9-d29e-42d2-8221-2fdd9db7f7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:08.352709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:08.361013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-2ece46c9-d29e-42d2-8221-2fdd9db7f7d1 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:08.364302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2ece46c9-d29e-42d2-8221-2fdd9db7f7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:08.470867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d36ae7-57bb-47fb-bca6-03bba94516d8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:08.505030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 235/486] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:15:08.526516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b58253-8f4d-4561-b146-2ee9947bc643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:08.527157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b58253-8f4d-4561-b146-2ee9947bc643 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:08.527440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b58253-8f4d-4561-b146-2ee9947bc643 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:08.527747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b58253-8f4d-4561-b146-2ee9947bc643 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:08.543190 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b58253-8f4d-4561-b146-2ee9947bc643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:08.544695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b58253-8f4d-4561-b146-2ee9947bc643 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:08.557539 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc778d-aade-4ced-bd71-ad4c0d2a4cf3 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:08.557821 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc778d-aade-4ced-bd71-ad4c0d2a4cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:08.558098 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc778d-aade-4ced-bd71-ad4c0d2a4cf3 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:08.559431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 236/487] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a7b58253-8f4d-4561-b146-2ee9947bc643 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:08.567783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:08.568764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:08.569699 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:08.569699 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:08.569699 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:08.569699 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 252/488] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:08.572982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28590120-caea-496a-92bb-a42740a67704 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:08.573267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28590120-caea-496a-92bb-a42740a67704 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:08.573547 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28590120-caea-496a-92bb-a42740a67704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:08.581151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b58253-8f4d-4561-b146-2ee9947bc643 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:08.608614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-28590120-caea-496a-92bb-a42740a67704 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:08.609839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-28590120-caea-496a-92bb-a42740a67704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:08.714611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:08.715897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:08.726540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b58253-8f4d-4561-b146-2ee9947bc643 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:08.754199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 237/489] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:08.791160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b2c63-1a28-425c-9f67-96a8e8be2ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:08.791812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b2c63-1a28-425c-9f67-96a8e8be2ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:08.792095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b2c63-1a28-425c-9f67-96a8e8be2ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:08.792368 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b2c63-1a28-425c-9f67-96a8e8be2ac3 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:08.793024 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b2c63-1a28-425c-9f67-96a8e8be2ac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:08.793330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b2c63-1a28-425c-9f67-96a8e8be2ac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:08.794701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b2c63-1a28-425c-9f67-96a8e8be2ac3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:08.813049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 253/490] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:08.817669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28590120-caea-496a-92bb-a42740a67704 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:08.822299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b2c63-1a28-425c-9f67-96a8e8be2ac3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:08.940024 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b2c63-1a28-425c-9f67-96a8e8be2ac3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:08.979904 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a1c90-0a2d-41c1-9366-416c939b3dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:08.980538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a1c90-0a2d-41c1-9366-416c939b3dfc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:08.980886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a1c90-0a2d-41c1-9366-416c939b3dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:08.981230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a1c90-0a2d-41c1-9366-416c939b3dfc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:08.988470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a1c90-0a2d-41c1-9366-416c939b3dfc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:08.989993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a1c90-0a2d-41c1-9366-416c939b3dfc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b38a1c90-0a2d-41c1-9366-416c939b3dfc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:09.002850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:09.003884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:09.004768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:09.004768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:09.004768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:09.004768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:09.004768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:09.004768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:09.004768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:09.004768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:09.004768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:15:09.004768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:09.004768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 254/491] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Sep 12 23:15:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:15:09.012046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a1c90-0a2d-41c1-9366-416c939b3dfc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:09.024486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:09.057139 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28590120-caea-496a-92bb-a42740a67704 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:09.095058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc690d2-a5df-4b10-b8fc-43574f1c3123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:09.095967 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc690d2-a5df-4b10-b8fc-43574f1c3123 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:09.096253 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc690d2-a5df-4b10-b8fc-43574f1c3123 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:09.096527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc690d2-a5df-4b10-b8fc-43574f1c3123 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:09.111184 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc690d2-a5df-4b10-b8fc-43574f1c3123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-8bc690d2-a5df-4b10-b8fc-43574f1c3123 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:09.116118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:09.117037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:09.118513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:09.118513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:09.118513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:09.118513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:09.118513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:09.118513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:09.118513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:09.118513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:09.118513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:09.118513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:09.118513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:09.118513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:09.118513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 238/492] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:09.118513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc690d2-a5df-4b10-b8fc-43574f1c3123 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:09.162048 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a1c90-0a2d-41c1-9366-416c939b3dfc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:09.172890 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:09.208065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3467e-3a1c-4bd5-b24a-8fd9f89af7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:09.209444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3467e-3a1c-4bd5-b24a-8fd9f89af7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:09.209933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3467e-3a1c-4bd5-b24a-8fd9f89af7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:09.210430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3467e-3a1c-4bd5-b24a-8fd9f89af7a0 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:09.212316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3467e-3a1c-4bd5-b24a-8fd9f89af7a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:09.212836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3467e-3a1c-4bd5-b24a-8fd9f89af7a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:09.215034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3467e-3a1c-4bd5-b24a-8fd9f89af7a0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:09.236083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc690d2-a5df-4b10-b8fc-43574f1c3123 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:09.260242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 255/493] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Sep 12 23:15:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:09.265525 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3467e-3a1c-4bd5-b24a-8fd9f89af7a0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:09.273405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8e790-554b-4e5a-b936-c51d35ec1817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:09.274496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8e790-554b-4e5a-b936-c51d35ec1817 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:09.274781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8e790-554b-4e5a-b936-c51d35ec1817 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:09.275053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8e790-554b-4e5a-b936-c51d35ec1817 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:09.286471 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8e790-554b-4e5a-b936-c51d35ec1817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-a5c8e790-554b-4e5a-b936-c51d35ec1817 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:09.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:09.292729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:09.293514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:09.293514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:09.293514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:09.293514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:09.293514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:09.293514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:09.293514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:09.293514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:09.293514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:09.293514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:09.293514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:09.293514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:09.293514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 239/494] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:09.294201 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1121a-a853-408b-abd6-7f27e5a38efa None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:09.294749 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1121a-a853-408b-abd6-7f27e5a38efa None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:09.294749 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1121a-a853-408b-abd6-7f27e5a38efa None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:09.300186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 240/495] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:09.300186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1121a-a853-408b-abd6-7f27e5a38efa None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:09.340186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:09.377718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3467e-3a1c-4bd5-b24a-8fd9f89af7a0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:09.404974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1121a-a853-408b-abd6-7f27e5a38efa None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:09.406608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:09.418328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf12f4d-2fbe-4781-ab55-49cc1f2e6b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:09.418932 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf12f4d-2fbe-4781-ab55-49cc1f2e6b6f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:09.419217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf12f4d-2fbe-4781-ab55-49cc1f2e6b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:09.419498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf12f4d-2fbe-4781-ab55-49cc1f2e6b6f None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:09.420183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf12f4d-2fbe-4781-ab55-49cc1f2e6b6f None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:09.420498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf12f4d-2fbe-4781-ab55-49cc1f2e6b6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:09.421869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf12f4d-2fbe-4781-ab55-49cc1f2e6b6f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:09.430866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 256/496] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:09.431492 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf12f4d-2fbe-4781-ab55-49cc1f2e6b6f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:09.444827 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07e6cc-62f0-4a53-99b0-c6d872bb5a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:09.445440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07e6cc-62f0-4a53-99b0-c6d872bb5a72 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:09.445721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07e6cc-62f0-4a53-99b0-c6d872bb5a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:09.446086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07e6cc-62f0-4a53-99b0-c6d872bb5a72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:09.517313 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07e6cc-62f0-4a53-99b0-c6d872bb5a72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:09.518851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07e6cc-62f0-4a53-99b0-c6d872bb5a72 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:09.542013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf12f4d-2fbe-4781-ab55-49cc1f2e6b6f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:09.570332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:09.577274 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c9b51-91d4-4ffc-b00b-8c0bc8e38b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:09.577815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c9b51-91d4-4ffc-b00b-8c0bc8e38b49 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:09.578095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c9b51-91d4-4ffc-b00b-8c0bc8e38b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:09.578367 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c9b51-91d4-4ffc-b00b-8c0bc8e38b49 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:09.579020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c9b51-91d4-4ffc-b00b-8c0bc8e38b49 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:09.579324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c9b51-91d4-4ffc-b00b-8c0bc8e38b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:09.580754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c9b51-91d4-4ffc-b00b-8c0bc8e38b49 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:09.589742 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 257/497] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:09.590738 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc184e3c-195d-43a5-a153-5acd0464bfcd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:09.591022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc184e3c-195d-43a5-a153-5acd0464bfcd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:09.591292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc184e3c-195d-43a5-a153-5acd0464bfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:09.610351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dc184e3c-195d-43a5-a153-5acd0464bfcd None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:09.612025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dc184e3c-195d-43a5-a153-5acd0464bfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:09.627036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 241/498] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:15:09.627856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07e6cc-62f0-4a53-99b0-c6d872bb5a72 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:09.630261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:09.690582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 258/499] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:09.691490 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc184e3c-195d-43a5-a153-5acd0464bfcd None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:09.745219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07e6cc-62f0-4a53-99b0-c6d872bb5a72 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:09.790909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f0b5e-b175-4b2a-9a27-faf107985823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:09.791553 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f0b5e-b175-4b2a-9a27-faf107985823 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:09.791876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f0b5e-b175-4b2a-9a27-faf107985823 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:09.792151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f0b5e-b175-4b2a-9a27-faf107985823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:09.802588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f0b5e-b175-4b2a-9a27-faf107985823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:09.804096 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f0b5e-b175-4b2a-9a27-faf107985823 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:09.809671 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc184e3c-195d-43a5-a153-5acd0464bfcd None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-4d1f0b5e-b175-4b2a-9a27-faf107985823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:09.819003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:09.820009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:09.821493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:09.821493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:09.821493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:09.823705 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 242/500] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:09.824960 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f0b5e-b175-4b2a-9a27-faf107985823 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:09.835739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:09.849779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93bca9-a7cd-4d31-be96-27802d5158c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:09.850403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93bca9-a7cd-4d31-be96-27802d5158c6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:09.850756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93bca9-a7cd-4d31-be96-27802d5158c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:09.850949 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93bca9-a7cd-4d31-be96-27802d5158c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:09.861062 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93bca9-a7cd-4d31-be96-27802d5158c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:09.862505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93bca9-a7cd-4d31-be96-27802d5158c6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-fa93bca9-a7cd-4d31-be96-27802d5158c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:09.877599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:09.878482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:09.879335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:09.879335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:09.879335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:09.879335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 259/501] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:09.879929 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ace8e-5597-4053-82ac-c1412ded146c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:09.880264 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ace8e-5597-4053-82ac-c1412ded146c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:09.880540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ace8e-5597-4053-82ac-c1412ded146c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:09.886260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 260/502] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:09.886260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ace8e-5597-4053-82ac-c1412ded146c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:09.891539 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:09.941855 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f0b5e-b175-4b2a-9a27-faf107985823 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:09.975759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe8a8e-789a-4573-8b35-f0836f610f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:09.976393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe8a8e-789a-4573-8b35-f0836f610f6a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:09.976674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe8a8e-789a-4573-8b35-f0836f610f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:09.976948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe8a8e-789a-4573-8b35-f0836f610f6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:09.987247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe8a8e-789a-4573-8b35-f0836f610f6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:09.988731 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe8a8e-789a-4573-8b35-f0836f610f6a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:09.997504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ace8e-5597-4053-82ac-c1412ded146c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-57fe8a8e-789a-4573-8b35-f0836f610f6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:10.002826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:10.003789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:10.004633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:10.004633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:10.004633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:10.004633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 243/503] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:10.004865 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe8a8e-789a-4573-8b35-f0836f610f6a None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:10.034466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e7d23-b736-4187-aea7-53aa5f5b16bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:10.035068 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e7d23-b736-4187-aea7-53aa5f5b16bf None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:10.035351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e7d23-b736-4187-aea7-53aa5f5b16bf None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:10.035653 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e7d23-b736-4187-aea7-53aa5f5b16bf None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:10.036289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e7d23-b736-4187-aea7-53aa5f5b16bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:10.036595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e7d23-b736-4187-aea7-53aa5f5b16bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:10.037957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e7d23-b736-4187-aea7-53aa5f5b16bf None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:10.049936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 261/504] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:10.051280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9d3a9-edc4-4aae-a646-3d21c550bfaa None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:10.051586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9d3a9-edc4-4aae-a646-3d21c550bfaa None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:10.051866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9d3a9-edc4-4aae-a646-3d21c550bfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:10.068823 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c1c9d3a9-edc4-4aae-a646-3d21c550bfaa None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:10.070008 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c1c9d3a9-edc4-4aae-a646-3d21c550bfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:10.096621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:10.106812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe8a8e-789a-4573-8b35-f0836f610f6a None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:10.114246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:10.136805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 262/505] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:10.137432 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9d3a9-edc4-4aae-a646-3d21c550bfaa None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:10.145534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571eafd-c682-421a-a2b1-882e732fac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:10.146131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571eafd-c682-421a-a2b1-882e732fac87 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:10.146405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571eafd-c682-421a-a2b1-882e732fac87 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:10.146670 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571eafd-c682-421a-a2b1-882e732fac87 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:10.147300 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571eafd-c682-421a-a2b1-882e732fac87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:10.147625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571eafd-c682-421a-a2b1-882e732fac87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:10.149510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571eafd-c682-421a-a2b1-882e732fac87 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:10.162671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 244/506] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:10.167900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571eafd-c682-421a-a2b1-882e732fac87 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:10.249945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9d3a9-edc4-4aae-a646-3d21c550bfaa None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:10.270951 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571eafd-c682-421a-a2b1-882e732fac87 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:10.287482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e963e25-63f3-4e7e-bd8a-178ecadb9963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:10.288148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e963e25-63f3-4e7e-bd8a-178ecadb9963 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:10.288440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e963e25-63f3-4e7e-bd8a-178ecadb9963 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:10.288708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e963e25-63f3-4e7e-bd8a-178ecadb9963 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:10.299243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e963e25-63f3-4e7e-bd8a-178ecadb9963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5e963e25-63f3-4e7e-bd8a-178ecadb9963 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:10.304168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:10.305235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:10.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:10.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:10.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:10.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:10.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:10.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:10.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:10.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:10.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:10.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:10.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:10.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:10.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 263/507] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:10.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e963e25-63f3-4e7e-bd8a-178ecadb9963 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:10.311748 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edc3af-0ded-4318-b1f9-829383aab435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:10.312421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edc3af-0ded-4318-b1f9-829383aab435 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:10.312708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edc3af-0ded-4318-b1f9-829383aab435 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:10.312978 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edc3af-0ded-4318-b1f9-829383aab435 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:10.313614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edc3af-0ded-4318-b1f9-829383aab435 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:10.313915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edc3af-0ded-4318-b1f9-829383aab435 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:10.315277 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edc3af-0ded-4318-b1f9-829383aab435 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:10.327624 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 245/508] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:10.328783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edc3af-0ded-4318-b1f9-829383aab435 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:10.335137 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:10.363012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:10.412373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e963e25-63f3-4e7e-bd8a-178ecadb9963 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:10.437561 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edc3af-0ded-4318-b1f9-829383aab435 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:10.447666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b432a-1f08-4be4-bbca-d3d71c445075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:10.448297 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b432a-1f08-4be4-bbca-d3d71c445075 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:10.448584 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b432a-1f08-4be4-bbca-d3d71c445075 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:10.448867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b432a-1f08-4be4-bbca-d3d71c445075 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:10.456282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b432a-1f08-4be4-bbca-d3d71c445075 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:10.457723 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b432a-1f08-4be4-bbca-d3d71c445075 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ec2b432a-1f08-4be4-bbca-d3d71c445075 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:10.469250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:10.470106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:10.470983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:10.470983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:10.470983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:10.470983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:10.470983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:10.470983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:10.470983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:10.470983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:10.470983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 23:15:10.470983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:10.470983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 264/509] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:15:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:10.471649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b432a-1f08-4be4-bbca-d3d71c445075 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:10.476137 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a678526-f9a7-45ef-b624-222186ee5fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:10.476766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a678526-f9a7-45ef-b624-222186ee5fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:10.477071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a678526-f9a7-45ef-b624-222186ee5fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:10.477348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a678526-f9a7-45ef-b624-222186ee5fd3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:10.484944 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a678526-f9a7-45ef-b624-222186ee5fd3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:10.486414 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a678526-f9a7-45ef-b624-222186ee5fd3 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-6a678526-f9a7-45ef-b624-222186ee5fd3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:10.498902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:10.499859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:10.500772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:10.500772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:10.500772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:10.500772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:10.500772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:10.500772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:10.500772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:10.500772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:10.500772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:15:10.500772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:10.500772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 246/510] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:15:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:10.500772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a678526-f9a7-45ef-b624-222186ee5fd3 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:10.560283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:10.576017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b432a-1f08-4be4-bbca-d3d71c445075 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:10.587171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:10.603057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a678526-f9a7-45ef-b624-222186ee5fd3 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:10.609444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15957be5-ff12-466a-ba25-c5f8319efa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:10.610050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15957be5-ff12-466a-ba25-c5f8319efa27 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:10.610330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15957be5-ff12-466a-ba25-c5f8319efa27 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:10.610601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15957be5-ff12-466a-ba25-c5f8319efa27 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:10.611262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15957be5-ff12-466a-ba25-c5f8319efa27 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:10.611595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15957be5-ff12-466a-ba25-c5f8319efa27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:10.612957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15957be5-ff12-466a-ba25-c5f8319efa27 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:10.625802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 265/511] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:10.625802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15957be5-ff12-466a-ba25-c5f8319efa27 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:10.639885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1776d1-2b8c-41d3-bc4e-bd4df2d69d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:10.640515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1776d1-2b8c-41d3-bc4e-bd4df2d69d07 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:10.640795 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1776d1-2b8c-41d3-bc4e-bd4df2d69d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:10.641075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1776d1-2b8c-41d3-bc4e-bd4df2d69d07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:10.647543 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1776d1-2b8c-41d3-bc4e-bd4df2d69d07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:10.649542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1776d1-2b8c-41d3-bc4e-bd4df2d69d07 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-2d1776d1-2b8c-41d3-bc4e-bd4df2d69d07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:10.661402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:10.662230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:10.663111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:10.663111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:10.663111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:10.663111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:10.663111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:10.663111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:10.663111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:10.663111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:10.663111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 247/512] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:10.663695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1776d1-2b8c-41d3-bc4e-bd4df2d69d07 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:10.725347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15957be5-ff12-466a-ba25-c5f8319efa27 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:10.758171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e05e7-61d8-46f9-8882-7907bc7b099c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:10.759381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e05e7-61d8-46f9-8882-7907bc7b099c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:10.759698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e05e7-61d8-46f9-8882-7907bc7b099c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:10.759818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1776d1-2b8c-41d3-bc4e-bd4df2d69d07 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:10.759972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e05e7-61d8-46f9-8882-7907bc7b099c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:10.770418 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e05e7-61d8-46f9-8882-7907bc7b099c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:10.771888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e05e7-61d8-46f9-8882-7907bc7b099c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:10.778050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-589e05e7-61d8-46f9-8882-7907bc7b099c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:10.786868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:10.787895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:10.788902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:10.788902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:10.788902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:10.788902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 266/513] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:10.788902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e05e7-61d8-46f9-8882-7907bc7b099c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:10.796186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f69f3-ca79-4178-bca8-24e3917aab4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:10.796792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f69f3-ca79-4178-bca8-24e3917aab4f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:10.797077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f69f3-ca79-4178-bca8-24e3917aab4f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:10.797378 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f69f3-ca79-4178-bca8-24e3917aab4f None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:10.798104 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f69f3-ca79-4178-bca8-24e3917aab4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:10.798412 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f69f3-ca79-4178-bca8-24e3917aab4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:10.799873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f69f3-ca79-4178-bca8-24e3917aab4f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:10.836350 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 248/514] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Sep 12 23:15:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:10.837139 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f69f3-ca79-4178-bca8-24e3917aab4f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:10.841648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:10.887066 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e05e7-61d8-46f9-8882-7907bc7b099c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:10.922636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b3ff7-ded5-4ac4-b1d3-4c1e66138b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:10.923249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b3ff7-ded5-4ac4-b1d3-4c1e66138b24 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:10.923531 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b3ff7-ded5-4ac4-b1d3-4c1e66138b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:10.923841 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b3ff7-ded5-4ac4-b1d3-4c1e66138b24 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:10.924571 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b3ff7-ded5-4ac4-b1d3-4c1e66138b24 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:10.924885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b3ff7-ded5-4ac4-b1d3-4c1e66138b24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:10.926317 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b3ff7-ded5-4ac4-b1d3-4c1e66138b24 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:10.938622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f69f3-ca79-4178-bca8-24e3917aab4f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:10.943765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 267/515] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Sep 12 23:15:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:10.944575 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b3ff7-ded5-4ac4-b1d3-4c1e66138b24 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:10.972991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d532e2f-c0ac-48d2-a3f0-f9b0fbfc010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:10.973636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d532e2f-c0ac-48d2-a3f0-f9b0fbfc010e None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:10.973906 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d532e2f-c0ac-48d2-a3f0-f9b0fbfc010e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:10.974186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d532e2f-c0ac-48d2-a3f0-f9b0fbfc010e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:10.984795 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d532e2f-c0ac-48d2-a3f0-f9b0fbfc010e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:10.986255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d532e2f-c0ac-48d2-a3f0-f9b0fbfc010e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:10.998877 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-6d532e2f-c0ac-48d2-a3f0-f9b0fbfc010e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:11.000950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:11.001883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:11.002706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:11.002706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:11.002706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:11.002706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 249/516] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:11.002926 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d532e2f-c0ac-48d2-a3f0-f9b0fbfc010e None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:11.047248 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b3ff7-ded5-4ac4-b1d3-4c1e66138b24 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:11.057577 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:11.081411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b00aef-2f94-4235-8577-15557a3eb6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:11.082165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b00aef-2f94-4235-8577-15557a3eb6d5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:11.082446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b00aef-2f94-4235-8577-15557a3eb6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:11.082718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b00aef-2f94-4235-8577-15557a3eb6d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:11.083405 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b00aef-2f94-4235-8577-15557a3eb6d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:11.083741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b00aef-2f94-4235-8577-15557a3eb6d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:11.085211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b00aef-2f94-4235-8577-15557a3eb6d5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:11.101347 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d532e2f-c0ac-48d2-a3f0-f9b0fbfc010e None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:11.102959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 268/517] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:11.103775 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b00aef-2f94-4235-8577-15557a3eb6d5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:11.140328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea04c70-79d4-41c3-b78b-c6a7541edf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:11.141027 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea04c70-79d4-41c3-b78b-c6a7541edf98 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:11.141219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea04c70-79d4-41c3-b78b-c6a7541edf98 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:11.141495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea04c70-79d4-41c3-b78b-c6a7541edf98 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:11.142145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea04c70-79d4-41c3-b78b-c6a7541edf98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:11.142452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea04c70-79d4-41c3-b78b-c6a7541edf98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:11.143920 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea04c70-79d4-41c3-b78b-c6a7541edf98 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:11.156268 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 250/518] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:11.157200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea04c70-79d4-41c3-b78b-c6a7541edf98 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:11.226221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b00aef-2f94-4235-8577-15557a3eb6d5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:11.243488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:11.264170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4909535-8825-4e3f-ae20-5b3f0dd899ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:11.264779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4909535-8825-4e3f-ae20-5b3f0dd899ba None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:11.265065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4909535-8825-4e3f-ae20-5b3f0dd899ba None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:11.265597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4909535-8825-4e3f-ae20-5b3f0dd899ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:11.271506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea04c70-79d4-41c3-b78b-c6a7541edf98 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:11.290273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:11.309052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c85aec-b3df-428f-b28f-2aea119e2a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:11.309658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c85aec-b3df-428f-b28f-2aea119e2a45 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:11.309943 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c85aec-b3df-428f-b28f-2aea119e2a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:11.310224 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c85aec-b3df-428f-b28f-2aea119e2a45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:11.339696 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4909535-8825-4e3f-ae20-5b3f0dd899ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:11.341186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4909535-8825-4e3f-ae20-5b3f0dd899ba None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:11.384059 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c85aec-b3df-428f-b28f-2aea119e2a45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:11.385454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c85aec-b3df-428f-b28f-2aea119e2a45 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:11.444545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 269/519] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:15:11.445392 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4909535-8825-4e3f-ae20-5b3f0dd899ba None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:11.464482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:11.484809 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 251/520] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:15:11.485691 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c85aec-b3df-428f-b28f-2aea119e2a45 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:11.506846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:11.549048 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4909535-8825-4e3f-ae20-5b3f0dd899ba None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:11.583099 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6e594-a1bf-4cba-8baf-6a88f6967ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:11.583778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6e594-a1bf-4cba-8baf-6a88f6967ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:11.584011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6e594-a1bf-4cba-8baf-6a88f6967ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:11.584314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6e594-a1bf-4cba-8baf-6a88f6967ca8 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:11.584952 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6e594-a1bf-4cba-8baf-6a88f6967ca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:11.585258 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6e594-a1bf-4cba-8baf-6a88f6967ca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:11.587164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6e594-a1bf-4cba-8baf-6a88f6967ca8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:11.588008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c85aec-b3df-428f-b28f-2aea119e2a45 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:11.598435 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 270/521] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:11.599136 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6e594-a1bf-4cba-8baf-6a88f6967ca8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:11.626426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb9364-f594-41d2-a023-27d773218682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:11.627021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb9364-f594-41d2-a023-27d773218682 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:11.627301 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb9364-f594-41d2-a023-27d773218682 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:11.627593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb9364-f594-41d2-a023-27d773218682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:11.628314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb9364-f594-41d2-a023-27d773218682 None admin] RBAC: Adding query filter params (role.id=5c7e1f557ae04250a01db783f3c27a5f, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=f5fa28ffdafd435cb2d93bd453add3cb) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:11.628622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb9364-f594-41d2-a023-27d773218682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:11.630048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb9364-f594-41d2-a023-27d773218682 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:11.635484 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 252/522] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:11 2022] GET /identity/v3/role_assignments?user.id=f5fa28ffdafd435cb2d93bd453add3cb&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=5c7e1f557ae04250a01db783f3c27a5f => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:11.636078 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb9364-f594-41d2-a023-27d773218682 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:11.689146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:11.716789 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6e594-a1bf-4cba-8baf-6a88f6967ca8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:11.733839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:11.753476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb9364-f594-41d2-a023-27d773218682 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:11.759809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b45a3-5834-41c9-940a-aecec2f3ee15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:11.760434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b45a3-5834-41c9-940a-aecec2f3ee15 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:11.760714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b45a3-5834-41c9-940a-aecec2f3ee15 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:11.760988 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b45a3-5834-41c9-940a-aecec2f3ee15 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:11.762273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b45a3-5834-41c9-940a-aecec2f3ee15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:11.764222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b45a3-5834-41c9-940a-aecec2f3ee15 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:11.788648 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 271/523] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:15:11 2022] POST /identity/v3/users => generated 247 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:15:11.789295 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b45a3-5834-41c9-940a-aecec2f3ee15 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:11.802158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e29b5-5674-43ef-bbb8-930b03af890d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:11.804158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e29b5-5674-43ef-bbb8-930b03af890d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:11.806348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e29b5-5674-43ef-bbb8-930b03af890d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:11.806966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e29b5-5674-43ef-bbb8-930b03af890d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:11.829610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e29b5-5674-43ef-bbb8-930b03af890d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:11.831312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e29b5-5674-43ef-bbb8-930b03af890d None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-f40e29b5-5674-43ef-bbb8-930b03af890d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:11.849873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:11.851177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:11.852308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:11.852308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:11.852308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:11.857976 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 253/524] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:11.857976 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e29b5-5674-43ef-bbb8-930b03af890d None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:11.950514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b45a3-5834-41c9-940a-aecec2f3ee15 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:11.983283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e29b5-5674-43ef-bbb8-930b03af890d None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:11.992185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:12.003322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc7c6e-7518-443a-80da-08afc4f96bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:12.003322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc7c6e-7518-443a-80da-08afc4f96bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:12.003651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc7c6e-7518-443a-80da-08afc4f96bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:12.003765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc7c6e-7518-443a-80da-08afc4f96bb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:12.016008 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc7c6e-7518-443a-80da-08afc4f96bb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:12.017485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc7c6e-7518-443a-80da-08afc4f96bb3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:12.023817 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:12.030890 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc52d47-7e89-4f3d-b1e2-e21d082ca024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:12.031881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc52d47-7e89-4f3d-b1e2-e21d082ca024 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:12.032374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc52d47-7e89-4f3d-b1e2-e21d082ca024 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:12.033102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc52d47-7e89-4f3d-b1e2-e21d082ca024 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d7cc7c6e-7518-443a-80da-08afc4f96bb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:12.034840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:12.035768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:12.037115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:12.037115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:12.037115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:12.038032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 272/525] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:12.038979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc7c6e-7518-443a-80da-08afc4f96bb3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:12.047783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc52d47-7e89-4f3d-b1e2-e21d082ca024 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:12.049524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc52d47-7e89-4f3d-b1e2-e21d082ca024 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-0dc52d47-7e89-4f3d-b1e2-e21d082ca024 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:12.062715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:12.063681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:12.064590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:12.064590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:12.064590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:12.064590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:12.064590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:12.064590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:12.064590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:12.064590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:12.064590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 23:15:12.064590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:12.064590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 254/526] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:12.065163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc52d47-7e89-4f3d-b1e2-e21d082ca024 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:12.159813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc7c6e-7518-443a-80da-08afc4f96bb3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:12.181737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc52d47-7e89-4f3d-b1e2-e21d082ca024 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:12.224221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e56f58-a628-4218-b3df-0aa7fecadb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:12.224912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e56f58-a628-4218-b3df-0aa7fecadb67 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:12.225429 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e56f58-a628-4218-b3df-0aa7fecadb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:12.225749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e56f58-a628-4218-b3df-0aa7fecadb67 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:12.226824 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e56f58-a628-4218-b3df-0aa7fecadb67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:12.227141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e56f58-a628-4218-b3df-0aa7fecadb67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:12.228803 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e56f58-a628-4218-b3df-0aa7fecadb67 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:12.235404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928fdec-194d-4e73-a073-e8129f426e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:12.236365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928fdec-194d-4e73-a073-e8129f426e2c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:12.236880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928fdec-194d-4e73-a073-e8129f426e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:12.237353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928fdec-194d-4e73-a073-e8129f426e2c None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:12.238201 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928fdec-194d-4e73-a073-e8129f426e2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:12.238704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928fdec-194d-4e73-a073-e8129f426e2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:12.240316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928fdec-194d-4e73-a073-e8129f426e2c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:12.247791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 273/527] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:12.247791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e56f58-a628-4218-b3df-0aa7fecadb67 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:12.263074 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 255/528] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:12.264322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928fdec-194d-4e73-a073-e8129f426e2c None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:12.308090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:12.319770 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:12.366845 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e56f58-a628-4218-b3df-0aa7fecadb67 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:12.379452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928fdec-194d-4e73-a073-e8129f426e2c None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:12.404732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40882d33-9b2b-4e70-99f2-53cf3e6fd39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:12.405574 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40882d33-9b2b-4e70-99f2-53cf3e6fd39d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:12.405998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40882d33-9b2b-4e70-99f2-53cf3e6fd39d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:12.406502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40882d33-9b2b-4e70-99f2-53cf3e6fd39d None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:12.407456 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40882d33-9b2b-4e70-99f2-53cf3e6fd39d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:12.408356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40882d33-9b2b-4e70-99f2-53cf3e6fd39d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:12.410038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40882d33-9b2b-4e70-99f2-53cf3e6fd39d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:12.421052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9434da6-c100-4a93-a6d5-56f9a2f14016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:12.421685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9434da6-c100-4a93-a6d5-56f9a2f14016 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:12.421962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9434da6-c100-4a93-a6d5-56f9a2f14016 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:12.422230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9434da6-c100-4a93-a6d5-56f9a2f14016 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:12.428961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9434da6-c100-4a93-a6d5-56f9a2f14016 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:12.429842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 274/529] 10.222.0.27 () {56 vars in 1139 bytes} [Mon Sep 12 23:15:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:12.430456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9434da6-c100-4a93-a6d5-56f9a2f14016 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:12.431275 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40882d33-9b2b-4e70-99f2-53cf3e6fd39d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-f9434da6-c100-4a93-a6d5-56f9a2f14016 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:12.443223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:12.444122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:12.444989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:12.444989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:12.444989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:12.444989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:12.444989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:12.444989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:12.444989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:12.444989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:12.444989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 256/530] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:12.445455 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9434da6-c100-4a93-a6d5-56f9a2f14016 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:12.548834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9434da6-c100-4a93-a6d5-56f9a2f14016 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:12.557529 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40882d33-9b2b-4e70-99f2-53cf3e6fd39d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:12.559733 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:12.569808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:12.593428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fd6e1-d495-4903-89a6-c2153718b653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:12.594127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fd6e1-d495-4903-89a6-c2153718b653 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:12.594409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fd6e1-d495-4903-89a6-c2153718b653 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:12.594679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fd6e1-d495-4903-89a6-c2153718b653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:12.607253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52fce6-e926-4813-8180-84a285306091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:12.607968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52fce6-e926-4813-8180-84a285306091 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:12.608247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52fce6-e926-4813-8180-84a285306091 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:12.608615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52fce6-e926-4813-8180-84a285306091 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:12.616465 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52fce6-e926-4813-8180-84a285306091 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:12.618514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52fce6-e926-4813-8180-84a285306091 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ca52fce6-e926-4813-8180-84a285306091 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:12.632408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:12.633474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:12.634505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:12.634505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:12.634505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:12.634505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:12.634505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:12.634505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:12.634505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:12.634505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:12.634505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 275/531] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:12.634974 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52fce6-e926-4813-8180-84a285306091 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:12.680980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fd6e1-d495-4903-89a6-c2153718b653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:12.683322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fd6e1-d495-4903-89a6-c2153718b653 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:12.751961 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52fce6-e926-4813-8180-84a285306091 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:12.792676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a70ec6-3604-4a71-ad3c-827673cc1630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:12.793502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a70ec6-3604-4a71-ad3c-827673cc1630 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:12.793951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a70ec6-3604-4a71-ad3c-827673cc1630 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:12.794371 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a70ec6-3604-4a71-ad3c-827673cc1630 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:12.795188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a70ec6-3604-4a71-ad3c-827673cc1630 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:12.795644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a70ec6-3604-4a71-ad3c-827673cc1630 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:12.797203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a70ec6-3604-4a71-ad3c-827673cc1630 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:12.797478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 257/532] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:15:12.798418 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fd6e1-d495-4903-89a6-c2153718b653 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:12.813993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 276/533] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:12.815032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a70ec6-3604-4a71-ad3c-827673cc1630 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:12.816270 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:12.842768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:12.934150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fd6e1-d495-4903-89a6-c2153718b653 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:12.938856 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a70ec6-3604-4a71-ad3c-827673cc1630 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:12.975300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e491b-00ad-40d8-9d50-59c078442488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:12.975967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e491b-00ad-40d8-9d50-59c078442488 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:12.976234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e491b-00ad-40d8-9d50-59c078442488 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:12.976510 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e491b-00ad-40d8-9d50-59c078442488 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:12.981128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538c64c-2998-4703-8e42-5503259e6ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:12.981896 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538c64c-2998-4703-8e42-5503259e6ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:12.982324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538c64c-2998-4703-8e42-5503259e6ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:12.982756 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538c64c-2998-4703-8e42-5503259e6ea4 None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:12.983617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538c64c-2998-4703-8e42-5503259e6ea4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:12.984578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538c64c-2998-4703-8e42-5503259e6ea4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:12.986663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538c64c-2998-4703-8e42-5503259e6ea4 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:12.987927 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e491b-00ad-40d8-9d50-59c078442488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:12.989528 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e491b-00ad-40d8-9d50-59c078442488 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4c5e491b-00ad-40d8-9d50-59c078442488 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:13.006288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:13.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:13.008413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:13.008413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:13.008413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:13.008413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 277/534] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:13.008413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e491b-00ad-40d8-9d50-59c078442488 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:13.010270 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 258/535] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:13.019889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538c64c-2998-4703-8e42-5503259e6ea4 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:13.093566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:13.112453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:13.122607 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e491b-00ad-40d8-9d50-59c078442488 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:13.152269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538c64c-2998-4703-8e42-5503259e6ea4 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:13.164073 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ba34f-8efe-4a77-9983-b0521cf62341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:13.165120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ba34f-8efe-4a77-9983-b0521cf62341 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:13.165401 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ba34f-8efe-4a77-9983-b0521cf62341 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:13.165672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ba34f-8efe-4a77-9983-b0521cf62341 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:13.166970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ba34f-8efe-4a77-9983-b0521cf62341 None admin] RBAC: Adding query filter params (role.id=5c7e1f557ae04250a01db783f3c27a5f, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=d61407406ef242a78030f79532fe2247) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:13.167285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ba34f-8efe-4a77-9983-b0521cf62341 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:13.168731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ba34f-8efe-4a77-9983-b0521cf62341 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:13.174674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 278/536] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:13 2022] GET /identity/v3/role_assignments?user.id=d61407406ef242a78030f79532fe2247&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=5c7e1f557ae04250a01db783f3c27a5f => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:13.175545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ba34f-8efe-4a77-9983-b0521cf62341 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:13.194895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ab328-206b-4b1b-9dd6-af562b36bcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:13.195864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ab328-206b-4b1b-9dd6-af562b36bcde None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:13.196152 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ab328-206b-4b1b-9dd6-af562b36bcde None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:13.196425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ab328-206b-4b1b-9dd6-af562b36bcde None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:13.197229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ab328-206b-4b1b-9dd6-af562b36bcde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:13.197550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ab328-206b-4b1b-9dd6-af562b36bcde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:13.199346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ab328-206b-4b1b-9dd6-af562b36bcde None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:13.219781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 259/537] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:13.221550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ab328-206b-4b1b-9dd6-af562b36bcde None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:13.365981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ab328-206b-4b1b-9dd6-af562b36bcde None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:13.397507 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ba34f-8efe-4a77-9983-b0521cf62341 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:13.397716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:13.405620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c4c97-f86b-433d-a0bd-bbc631d3bafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:13.406258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4c97-f86b-433d-a0bd-bbc631d3bafc None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:13.406552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4c97-f86b-433d-a0bd-bbc631d3bafc None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:13.406806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4c97-f86b-433d-a0bd-bbc631d3bafc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:13.416852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c4c97-f86b-433d-a0bd-bbc631d3bafc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:13.418691 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c4c97-f86b-433d-a0bd-bbc631d3bafc None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-922c4c97-f86b-433d-a0bd-bbc631d3bafc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:13.434529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:13.435404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:13.436267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:13.436267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:13.436267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:13.436267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:13.436267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:13.436267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:13.436267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:13.436267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:13.436267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 260/538] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:13.437796 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3c0b4-ec27-4257-854b-6c56926d4414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:13.438649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3c0b4-ec27-4257-854b-6c56926d4414 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:13.439121 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3c0b4-ec27-4257-854b-6c56926d4414 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:13.439618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3c0b4-ec27-4257-854b-6c56926d4414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:13.440540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3c0b4-ec27-4257-854b-6c56926d4414 None admin] RBAC: Adding query filter params (role.id=5c7e1f557ae04250a01db783f3c27a5f, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=c5b8828077594349a93c7a842c0f15da) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:13.441042 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3c0b4-ec27-4257-854b-6c56926d4414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:13.441453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c4c97-f86b-433d-a0bd-bbc631d3bafc None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:13.442905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3c0b4-ec27-4257-854b-6c56926d4414 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:13.453909 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 279/539] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:13 2022] GET /identity/v3/role_assignments?user.id=c5b8828077594349a93c7a842c0f15da&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=5c7e1f557ae04250a01db783f3c27a5f => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:13.562496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c4c97-f86b-433d-a0bd-bbc631d3bafc None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:13.612611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34571d49-ddd0-4715-94a7-a68305a86023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:13.613616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34571d49-ddd0-4715-94a7-a68305a86023 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:13.614099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34571d49-ddd0-4715-94a7-a68305a86023 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:13.614567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34571d49-ddd0-4715-94a7-a68305a86023 None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:13.615466 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34571d49-ddd0-4715-94a7-a68305a86023 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:13.616044 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34571d49-ddd0-4715-94a7-a68305a86023 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:13.618274 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34571d49-ddd0-4715-94a7-a68305a86023 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:13.635354 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 261/540] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:13.641506 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3c0b4-ec27-4257-854b-6c56926d4414 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:13.646035 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:13.760715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3c0b4-ec27-4257-854b-6c56926d4414 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:13.813138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313edae6-df48-471d-a6d5-90628db6201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:13.813781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313edae6-df48-471d-a6d5-90628db6201a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:13.814065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313edae6-df48-471d-a6d5-90628db6201a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:13.814340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313edae6-df48-471d-a6d5-90628db6201a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:13.815083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313edae6-df48-471d-a6d5-90628db6201a None admin] RBAC: Adding query filter params (role.id=5c7e1f557ae04250a01db783f3c27a5f, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=bc6e63e9fc3447d5a894a777a98fe773) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:13.815397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313edae6-df48-471d-a6d5-90628db6201a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:13.816883 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313edae6-df48-471d-a6d5-90628db6201a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:13.823719 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 280/541] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:13 2022] GET /identity/v3/role_assignments?user.id=bc6e63e9fc3447d5a894a777a98fe773&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=5c7e1f557ae04250a01db783f3c27a5f => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:14.051958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da6f5c-b575-48fe-bc5b-b4ec1779f687 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:14.052308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da6f5c-b575-48fe-bc5b-b4ec1779f687 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:14.052529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da6f5c-b575-48fe-bc5b-b4ec1779f687 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:14.054133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 262/542] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:14.059445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5ff4-3596-4209-9304-28c5077a233d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:14.059761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5ff4-3596-4209-9304-28c5077a233d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:14.060039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5ff4-3596-4209-9304-28c5077a233d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:14.079364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dbed5ff4-3596-4209-9304-28c5077a233d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:14.080791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dbed5ff4-3596-4209-9304-28c5077a233d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:14.147841 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:14.157232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 281/543] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:14.270452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b2234-1cae-47b3-a682-66d66dbe05f4 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:14.270975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b2234-1cae-47b3-a682-66d66dbe05f4 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:14.271237 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b2234-1cae-47b3-a682-66d66dbe05f4 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:14.273001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 263/544] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:14.278854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0292e3-43f6-48e5-94f5-0144c4dbdbec None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:14.279148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0292e3-43f6-48e5-94f5-0144c4dbdbec None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:14.279409 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0292e3-43f6-48e5-94f5-0144c4dbdbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:14.309219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ab0292e3-43f6-48e5-94f5-0144c4dbdbec None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:14.310433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ab0292e3-43f6-48e5-94f5-0144c4dbdbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:14.384809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 282/545] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:15:14.535452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b2234-1cae-47b3-a682-66d66dbe05f4 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:14.598809 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:14.655157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b2234-1cae-47b3-a682-66d66dbe05f4 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:14.696393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2787a4-670b-40d3-b370-7d852b739a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:14.697377 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2787a4-670b-40d3-b370-7d852b739a4b None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:14.697869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2787a4-670b-40d3-b370-7d852b739a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:14.698335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2787a4-670b-40d3-b370-7d852b739a4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:14.709206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2787a4-670b-40d3-b370-7d852b739a4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-3f2787a4-670b-40d3-b370-7d852b739a4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:14.714440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:14.715292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:14.716124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:14.716124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:14.716124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:14.716124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:14.716124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:14.716124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:14.716124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:14.716124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:14.716124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:14.716124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:14.716124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:14.716124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:14.718635 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 264/546] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:14.724561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0292e3-43f6-48e5-94f5-0144c4dbdbec None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:14.835365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0292e3-43f6-48e5-94f5-0144c4dbdbec None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:14.869839 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:14.885010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e687a90-e9db-4e03-938f-d857ba14cb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:14.885937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e687a90-e9db-4e03-938f-d857ba14cb52 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:14.886460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e687a90-e9db-4e03-938f-d857ba14cb52 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:14.886938 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e687a90-e9db-4e03-938f-d857ba14cb52 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:14.887819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e687a90-e9db-4e03-938f-d857ba14cb52 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:14.888314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e687a90-e9db-4e03-938f-d857ba14cb52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:14.889858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e687a90-e9db-4e03-938f-d857ba14cb52 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:14.898557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 283/547] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:14.904336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2787a4-670b-40d3-b370-7d852b739a4b None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:14.919365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e687a90-e9db-4e03-938f-d857ba14cb52 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:15.093499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2787a4-670b-40d3-b370-7d852b739a4b None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:15.108305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e687a90-e9db-4e03-938f-d857ba14cb52 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:15.121581 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:15.159744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f855aca-9168-4c0b-ba4d-62897ea4466c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:15.160368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f855aca-9168-4c0b-ba4d-62897ea4466c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:15.160647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f855aca-9168-4c0b-ba4d-62897ea4466c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:15.160914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f855aca-9168-4c0b-ba4d-62897ea4466c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:15.167596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5a9bc-03a6-4dc6-a69f-7479a6a036be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:15.168228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5a9bc-03a6-4dc6-a69f-7479a6a036be None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:15.168506 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5a9bc-03a6-4dc6-a69f-7479a6a036be None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:15.168778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5a9bc-03a6-4dc6-a69f-7479a6a036be None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:15.181160 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f855aca-9168-4c0b-ba4d-62897ea4466c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:15.182625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f855aca-9168-4c0b-ba4d-62897ea4466c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:15.189668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5a9bc-03a6-4dc6-a69f-7479a6a036be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-bde5a9bc-03a6-4dc6-a69f-7479a6a036be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:15.194930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:15.196019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:15.196939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:15.196939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:15.196939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:15.196939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:15.196939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:15.196939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:15.196939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:15.196939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:15.196939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:15.196939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:15.196939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:15.196939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:15.199533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 284/548] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:15.205737 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5a9bc-03a6-4dc6-a69f-7479a6a036be None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-6f855aca-9168-4c0b-ba4d-62897ea4466c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:15.212550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:15.213893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:15.214898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:15.214898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:15.214898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:15.223515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 265/549] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:15.228694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f855aca-9168-4c0b-ba4d-62897ea4466c None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:15.287923 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:15.381813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5a9bc-03a6-4dc6-a69f-7479a6a036be None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:15.416788 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f855aca-9168-4c0b-ba4d-62897ea4466c None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:15.420360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8f389-114e-4230-aa7b-10f2ee01b69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:15.420979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8f389-114e-4230-aa7b-10f2ee01b69c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:15.421262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8f389-114e-4230-aa7b-10f2ee01b69c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:15.421538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8f389-114e-4230-aa7b-10f2ee01b69c None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:15.422203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8f389-114e-4230-aa7b-10f2ee01b69c None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:15.422565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8f389-114e-4230-aa7b-10f2ee01b69c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:15.424017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8f389-114e-4230-aa7b-10f2ee01b69c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:15.433398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 285/550] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:15.440510 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8f389-114e-4230-aa7b-10f2ee01b69c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:15.462313 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822a8b8-a5e0-4a95-8d84-705932cc514c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:15.463023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822a8b8-a5e0-4a95-8d84-705932cc514c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:15.463307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822a8b8-a5e0-4a95-8d84-705932cc514c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:15.463593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822a8b8-a5e0-4a95-8d84-705932cc514c None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:15.464252 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822a8b8-a5e0-4a95-8d84-705932cc514c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:15.464558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822a8b8-a5e0-4a95-8d84-705932cc514c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:15.466042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822a8b8-a5e0-4a95-8d84-705932cc514c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:15.478384 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 266/551] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:15.483345 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822a8b8-a5e0-4a95-8d84-705932cc514c None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:15.531381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:15.602349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8f389-114e-4230-aa7b-10f2ee01b69c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:15.613482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:15.644483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da603b71-3854-4d75-8c23-80b8db674a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:15.645045 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da603b71-3854-4d75-8c23-80b8db674a47 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:15.645328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da603b71-3854-4d75-8c23-80b8db674a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:15.645603 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da603b71-3854-4d75-8c23-80b8db674a47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:15.672591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da603b71-3854-4d75-8c23-80b8db674a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:15.674166 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da603b71-3854-4d75-8c23-80b8db674a47 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:15.685911 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822a8b8-a5e0-4a95-8d84-705932cc514c None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-da603b71-3854-4d75-8c23-80b8db674a47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:15.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:15.693816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:15.694682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:15.694682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:15.694682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:15.694682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 286/552] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:15.700467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da603b71-3854-4d75-8c23-80b8db674a47 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:15.747873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59289a9c-20c7-4b9b-999e-9c0fb4153d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:15.749595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59289a9c-20c7-4b9b-999e-9c0fb4153d98 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:15.749871 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59289a9c-20c7-4b9b-999e-9c0fb4153d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:15.750141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59289a9c-20c7-4b9b-999e-9c0fb4153d98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:15.769971 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59289a9c-20c7-4b9b-999e-9c0fb4153d98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:15.771546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59289a9c-20c7-4b9b-999e-9c0fb4153d98 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-59289a9c-20c7-4b9b-999e-9c0fb4153d98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:15.791620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:15.792611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:15.793571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:15.793571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:15.793571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:15.793571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:15.793571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:15.793571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:15.793571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:15.793571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:15.793571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:15:15.793571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:15.794692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 267/553] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Sep 12 23:15:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:15:15.802908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59289a9c-20c7-4b9b-999e-9c0fb4153d98 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:15.889861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da603b71-3854-4d75-8c23-80b8db674a47 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:15.934096 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:15.937112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a10d4-9b58-4b26-a003-b35b3ddd8766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:15.937790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a10d4-9b58-4b26-a003-b35b3ddd8766 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:15.938177 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a10d4-9b58-4b26-a003-b35b3ddd8766 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:15.938547 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a10d4-9b58-4b26-a003-b35b3ddd8766 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:15.939292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a10d4-9b58-4b26-a003-b35b3ddd8766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:15.939755 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a10d4-9b58-4b26-a003-b35b3ddd8766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:15.942129 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a10d4-9b58-4b26-a003-b35b3ddd8766 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:15.952368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59289a9c-20c7-4b9b-999e-9c0fb4153d98 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:15.955926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 287/554] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:15.957462 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796409ce-4da0-4ae1-aa12-eba6c0dabf2f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:15.957906 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796409ce-4da0-4ae1-aa12-eba6c0dabf2f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:15.958372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796409ce-4da0-4ae1-aa12-eba6c0dabf2f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:15.959967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 288/555] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:15.961760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796409ce-4da0-4ae1-aa12-eba6c0dabf2f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:15.987791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:16.001800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9402a8a-454e-447a-93e8-dd03a2458af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:16.002417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9402a8a-454e-447a-93e8-dd03a2458af8 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:16.002693 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9402a8a-454e-447a-93e8-dd03a2458af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:16.002960 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9402a8a-454e-447a-93e8-dd03a2458af8 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:16.003775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9402a8a-454e-447a-93e8-dd03a2458af8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:16.004082 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9402a8a-454e-447a-93e8-dd03a2458af8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:16.005661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9402a8a-454e-447a-93e8-dd03a2458af8 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:16.029337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 268/556] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Sep 12 23:15:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:16.032214 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611e573-874e-4e08-bc05-0b5311fa441d None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:16.032963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611e573-874e-4e08-bc05-0b5311fa441d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:16.033441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611e573-874e-4e08-bc05-0b5311fa441d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:16.067893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-8611e573-874e-4e08-bc05-0b5311fa441d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:16.069095 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8611e573-874e-4e08-bc05-0b5311fa441d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:16.099558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796409ce-4da0-4ae1-aa12-eba6c0dabf2f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:16.141832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 269/557] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:16.142729 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858aecf-e561-45f3-b715-7f9d274bc0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:16.142855 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611e573-874e-4e08-bc05-0b5311fa441d None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:16.143813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858aecf-e561-45f3-b715-7f9d274bc0af None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:16.144269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858aecf-e561-45f3-b715-7f9d274bc0af None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:16.144712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858aecf-e561-45f3-b715-7f9d274bc0af None admin] PATH_INFO: `/v3/users/swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:16.154741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858aecf-e561-45f3-b715-7f9d274bc0af None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:16.157276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858aecf-e561-45f3-b715-7f9d274bc0af None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a858aecf-e561-45f3-b715-7f9d274bc0af None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:16.173634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:16.175095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:16.176065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:16.176065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:16.176065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:16.176065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:16.176065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:16.176065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:16.176065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:16.176065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:16.176065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:15:16.176065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:16.177616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 289/558] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:15:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:15:16.183618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858aecf-e561-45f3-b715-7f9d274bc0af None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:16.259923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:16.303874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611e573-874e-4e08-bc05-0b5311fa441d None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:16.312653 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:16.333996 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858aecf-e561-45f3-b715-7f9d274bc0af None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:16.341006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661420b-201a-4095-bd90-72657459356c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:16.341831 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661420b-201a-4095-bd90-72657459356c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:16.342260 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661420b-201a-4095-bd90-72657459356c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:16.342533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661420b-201a-4095-bd90-72657459356c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:16.353417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661420b-201a-4095-bd90-72657459356c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:16.354974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661420b-201a-4095-bd90-72657459356c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-7661420b-201a-4095-bd90-72657459356c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:16.373895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:16.375212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:16.376351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:16.376351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:16.376351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:16.377363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaee002-875b-4f1f-9092-7a38953e8183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:16.378082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaee002-875b-4f1f-9092-7a38953e8183 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:16.378460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaee002-875b-4f1f-9092-7a38953e8183 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:16.378606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 270/559] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:16.378957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaee002-875b-4f1f-9092-7a38953e8183 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:16.379944 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaee002-875b-4f1f-9092-7a38953e8183 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:16.380488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaee002-875b-4f1f-9092-7a38953e8183 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:16.380674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb65297-5c2e-4360-a9cc-54c56bcec2a8 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:16.381163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb65297-5c2e-4360-a9cc-54c56bcec2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:16.381577 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb65297-5c2e-4360-a9cc-54c56bcec2a8 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:16.382474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaee002-875b-4f1f-9092-7a38953e8183 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:16.383742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 271/560] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:16.389028 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb65297-5c2e-4360-a9cc-54c56bcec2a8 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:16.406928 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 290/561] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Sep 12 23:15:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:16.408785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df553d-b0b1-49ee-86b3-5eca16e4e9a6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:16.409164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df553d-b0b1-49ee-86b3-5eca16e4e9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:16.409530 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df553d-b0b1-49ee-86b3-5eca16e4e9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:16.433421 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c4df553d-b0b1-49ee-86b3-5eca16e4e9a6 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:16.434553 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c4df553d-b0b1-49ee-86b3-5eca16e4e9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:16.502972 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb65297-5c2e-4360-a9cc-54c56bcec2a8 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:16.504640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 291/562] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:16.505483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df553d-b0b1-49ee-86b3-5eca16e4e9a6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:16.530246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:16.542654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a132d21-a87a-4683-be86-d28d682aef47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:16.543248 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a132d21-a87a-4683-be86-d28d682aef47 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:16.543529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a132d21-a87a-4683-be86-d28d682aef47 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:16.544478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a132d21-a87a-4683-be86-d28d682aef47 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:16.544478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a132d21-a87a-4683-be86-d28d682aef47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:16.544749 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a132d21-a87a-4683-be86-d28d682aef47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:16.546105 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a132d21-a87a-4683-be86-d28d682aef47 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:16.562637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 272/563] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:16.562637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a132d21-a87a-4683-be86-d28d682aef47 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:16.593113 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:16.642542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df553d-b0b1-49ee-86b3-5eca16e4e9a6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:16.694786 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715b21e-3044-42d8-b73e-10d23d29844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:16.695482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715b21e-3044-42d8-b73e-10d23d29844e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:16.696012 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715b21e-3044-42d8-b73e-10d23d29844e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:16.696429 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715b21e-3044-42d8-b73e-10d23d29844e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:16.707443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a132d21-a87a-4683-be86-d28d682aef47 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:16.745116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c074bac-d938-4c04-bb2a-fd5919d9544b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:16.746017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c074bac-d938-4c04-bb2a-fd5919d9544b None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:16.746301 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c074bac-d938-4c04-bb2a-fd5919d9544b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:16.746577 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c074bac-d938-4c04-bb2a-fd5919d9544b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:16.756849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c074bac-d938-4c04-bb2a-fd5919d9544b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-5c074bac-d938-4c04-bb2a-fd5919d9544b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:16.761675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:16.762611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:16.763441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:16.763441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:16.763441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:16.763441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:16.763441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:16.763441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:16.763441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:16.763441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:16.763441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:16.763441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:16.763441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:16.763441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:16.763441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 273/564] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:16.764170 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34141b4c-344f-4708-891b-dc64d239cbb1 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:16.764170 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34141b4c-344f-4708-891b-dc64d239cbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:16.764898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34141b4c-344f-4708-891b-dc64d239cbb1 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:16.765799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 274/565] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:16.766344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34141b4c-344f-4708-891b-dc64d239cbb1 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:16.779891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715b21e-3044-42d8-b73e-10d23d29844e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:16.781451 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715b21e-3044-42d8-b73e-10d23d29844e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:16.813777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:16.848225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:16.868984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34141b4c-344f-4708-891b-dc64d239cbb1 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:16.880058 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 292/566] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:15:16.880879 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715b21e-3044-42d8-b73e-10d23d29844e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:16.905810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26dbf8-2294-416a-b085-1b3c90fa6230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:16.906440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26dbf8-2294-416a-b085-1b3c90fa6230 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:16.906723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26dbf8-2294-416a-b085-1b3c90fa6230 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:16.907005 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26dbf8-2294-416a-b085-1b3c90fa6230 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:16.916721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26dbf8-2294-416a-b085-1b3c90fa6230 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:16.916721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26dbf8-2294-416a-b085-1b3c90fa6230 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-4e26dbf8-2294-416a-b085-1b3c90fa6230 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:16.927552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:16.928458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:16.934106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:16.934106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:16.934106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:16.934106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:16.934106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:16.934106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:16.934106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:16.934106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:16.934106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 275/567] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:16.934106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470135d-f065-4bc8-84c7-9ab3defb6122 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:16.934106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470135d-f065-4bc8-84c7-9ab3defb6122 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:16.934731 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470135d-f065-4bc8-84c7-9ab3defb6122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:16.947748 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-6470135d-f065-4bc8-84c7-9ab3defb6122 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:16.948876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6470135d-f065-4bc8-84c7-9ab3defb6122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:16.980426 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715b21e-3044-42d8-b73e-10d23d29844e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:17.013361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c44c41-548b-40a5-8f53-8a1bdb4f9314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:17.013547 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 276/568] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:17.014386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470135d-f065-4bc8-84c7-9ab3defb6122 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:17.016071 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c44c41-548b-40a5-8f53-8a1bdb4f9314 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:17.016351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c44c41-548b-40a5-8f53-8a1bdb4f9314 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:17.016634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c44c41-548b-40a5-8f53-8a1bdb4f9314 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:17.017965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c44c41-548b-40a5-8f53-8a1bdb4f9314 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:17.017965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c44c41-548b-40a5-8f53-8a1bdb4f9314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:17.019082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c44c41-548b-40a5-8f53-8a1bdb4f9314 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:17.033442 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 293/569] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:17.034490 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c44c41-548b-40a5-8f53-8a1bdb4f9314 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:17.053772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:17.072458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:17.122816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470135d-f065-4bc8-84c7-9ab3defb6122 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:17.144069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c44c41-548b-40a5-8f53-8a1bdb4f9314 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:17.156828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4763b-fbde-4333-97d6-b5959ad22a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:17.157457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4763b-fbde-4333-97d6-b5959ad22a31 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:17.158012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4763b-fbde-4333-97d6-b5959ad22a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:17.158286 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4763b-fbde-4333-97d6-b5959ad22a31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:17.168754 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4763b-fbde-4333-97d6-b5959ad22a31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-7dd4763b-fbde-4333-97d6-b5959ad22a31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:17.173636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:17.174656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:17.175528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:17.175528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:17.175528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:17.175528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:17.175528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:17.175528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:17.175528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:17.175528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:17.175528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:17.175528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:17.175528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:17.175528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:17.179656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 277/570] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:17.180603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4763b-fbde-4333-97d6-b5959ad22a31 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:17.181943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624382e-256a-460b-ac18-3dc7453a6e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:17.182575 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624382e-256a-460b-ac18-3dc7453a6e0d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:17.182856 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624382e-256a-460b-ac18-3dc7453a6e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:17.183128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624382e-256a-460b-ac18-3dc7453a6e0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:17.194167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624382e-256a-460b-ac18-3dc7453a6e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:17.196393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624382e-256a-460b-ac18-3dc7453a6e0d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c624382e-256a-460b-ac18-3dc7453a6e0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:17.212194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:17.213133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:17.214013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:17.214013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:17.214013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:17.214194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 294/571] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:17.214883 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624382e-256a-460b-ac18-3dc7453a6e0d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:17.282162 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:17.290112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4763b-fbde-4333-97d6-b5959ad22a31 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:17.304550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:17.320934 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624382e-256a-460b-ac18-3dc7453a6e0d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:17.324825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2096d80-23a6-49f5-8c27-f502e5d2fa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:17.325443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2096d80-23a6-49f5-8c27-f502e5d2fa69 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:17.325725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2096d80-23a6-49f5-8c27-f502e5d2fa69 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:17.325999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2096d80-23a6-49f5-8c27-f502e5d2fa69 None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:17.326711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2096d80-23a6-49f5-8c27-f502e5d2fa69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:17.327039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2096d80-23a6-49f5-8c27-f502e5d2fa69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:17.328585 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2096d80-23a6-49f5-8c27-f502e5d2fa69 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:17.339865 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 278/572] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:17.340571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2096d80-23a6-49f5-8c27-f502e5d2fa69 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:17.353819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaeaa56-7984-4377-8c49-9739f80c3c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:17.354441 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaeaa56-7984-4377-8c49-9739f80c3c7f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:17.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaeaa56-7984-4377-8c49-9739f80c3c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:17.355000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaeaa56-7984-4377-8c49-9739f80c3c7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:17.366079 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaeaa56-7984-4377-8c49-9739f80c3c7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:17.368094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaeaa56-7984-4377-8c49-9739f80c3c7f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-1eaeaa56-7984-4377-8c49-9739f80c3c7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:17.381801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:17.382792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:17.383757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:17.383757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:17.383757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:17.383757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 295/573] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:17.383757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaeaa56-7984-4377-8c49-9739f80c3c7f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:17.442069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2096d80-23a6-49f5-8c27-f502e5d2fa69 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:17.474507 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887222df-e624-4b3d-9cad-9c892806fba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:17.475042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887222df-e624-4b3d-9cad-9c892806fba0 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:17.475324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887222df-e624-4b3d-9cad-9c892806fba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:17.475620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887222df-e624-4b3d-9cad-9c892806fba0 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:17.476272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887222df-e624-4b3d-9cad-9c892806fba0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:17.476572 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887222df-e624-4b3d-9cad-9c892806fba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:17.478517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887222df-e624-4b3d-9cad-9c892806fba0 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:17.485391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaeaa56-7984-4377-8c49-9739f80c3c7f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:17.486268 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 279/574] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:17.487064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887222df-e624-4b3d-9cad-9c892806fba0 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:17.504338 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:17.519811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0326349-4648-4b5e-a103-0f8b638d6ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:17.520407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0326349-4648-4b5e-a103-0f8b638d6ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:17.520685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0326349-4648-4b5e-a103-0f8b638d6ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:17.520964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0326349-4648-4b5e-a103-0f8b638d6ea8 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:17.521594 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0326349-4648-4b5e-a103-0f8b638d6ea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:17.521893 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0326349-4648-4b5e-a103-0f8b638d6ea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:17.523239 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0326349-4648-4b5e-a103-0f8b638d6ea8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:17.526343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:17.536179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 296/575] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:17.540924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0326349-4648-4b5e-a103-0f8b638d6ea8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:17.586658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887222df-e624-4b3d-9cad-9c892806fba0 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:17.619762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6306a-0dbc-42da-831a-3a48a2ff2281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:17.620534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6306a-0dbc-42da-831a-3a48a2ff2281 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:17.620675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6306a-0dbc-42da-831a-3a48a2ff2281 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:17.620928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6306a-0dbc-42da-831a-3a48a2ff2281 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:17.630735 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f6306a-0dbc-42da-831a-3a48a2ff2281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-a4f6306a-0dbc-42da-831a-3a48a2ff2281 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:17.636083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:17.637057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:17.637888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:17.637888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:17.637888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:17.637888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:17.637888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:17.637888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:17.637888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:17.637888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:17.637888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:17.637888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:17.637888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:17.637888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:17.637888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 280/576] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:17.638591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6306a-0dbc-42da-831a-3a48a2ff2281 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:17.638659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0326349-4648-4b5e-a103-0f8b638d6ea8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:17.674216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab0e22-184d-4488-8794-b0ab0906e58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:17.674867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab0e22-184d-4488-8794-b0ab0906e58f None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:17.675146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab0e22-184d-4488-8794-b0ab0906e58f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:17.675421 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab0e22-184d-4488-8794-b0ab0906e58f None admin] PATH_INFO: `/v3/projects/32bbb936501846e8914acf07890c2891/users/f5fa28ffdafd435cb2d93bd453add3cb/roles/5c7e1f557ae04250a01db783f3c27a5f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:17.689415 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab0e22-184d-4488-8794-b0ab0906e58f None admin] RBAC: Authorizing `identity:create_grant(project_id=32bbb936501846e8914acf07890c2891, user_id=f5fa28ffdafd435cb2d93bd453add3cb, role_id=5c7e1f557ae04250a01db783f3c27a5f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:17.690831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab0e22-184d-4488-8794-b0ab0906e58f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:17.702940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 297/577] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:15:17 2022] PUT /identity/v3/projects/32bbb936501846e8914acf07890c2891/users/f5fa28ffdafd435cb2d93bd453add3cb/roles/5c7e1f557ae04250a01db783f3c27a5f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:15:17.703771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab0e22-184d-4488-8794-b0ab0906e58f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:17.718892 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:17.749294 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6306a-0dbc-42da-831a-3a48a2ff2281 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:17.759803 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:17.791621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae77ce1-072c-408c-9c9e-21fb31192d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:17.792561 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae77ce1-072c-408c-9c9e-21fb31192d51 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:17.792844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae77ce1-072c-408c-9c9e-21fb31192d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:17.793116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae77ce1-072c-408c-9c9e-21fb31192d51 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:17.793768 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae77ce1-072c-408c-9c9e-21fb31192d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:17.794075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae77ce1-072c-408c-9c9e-21fb31192d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:17.796133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae77ce1-072c-408c-9c9e-21fb31192d51 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:17.814898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 281/578] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:17.814898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae77ce1-072c-408c-9c9e-21fb31192d51 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:17.830364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab0e22-184d-4488-8794-b0ab0906e58f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:17.867107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adbd7c-a1bd-44ef-b1c2-738e153bba48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:17.868392 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adbd7c-a1bd-44ef-b1c2-738e153bba48 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:17.868859 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adbd7c-a1bd-44ef-b1c2-738e153bba48 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:17.869136 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adbd7c-a1bd-44ef-b1c2-738e153bba48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:17.882199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adbd7c-a1bd-44ef-b1c2-738e153bba48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:17.883712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adbd7c-a1bd-44ef-b1c2-738e153bba48 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f0adbd7c-a1bd-44ef-b1c2-738e153bba48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:17.899680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:17.900668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:17.901557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:17.901557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:17.901557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:17.901557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 298/579] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:17.902393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adbd7c-a1bd-44ef-b1c2-738e153bba48 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:17.938587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae77ce1-072c-408c-9c9e-21fb31192d51 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:17.973672 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf406c41-934e-4734-9e2a-1237ecea77c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:17.974299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf406c41-934e-4734-9e2a-1237ecea77c6 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:17.974578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf406c41-934e-4734-9e2a-1237ecea77c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:17.974853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf406c41-934e-4734-9e2a-1237ecea77c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:17.979083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:17.983990 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf406c41-934e-4734-9e2a-1237ecea77c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:17.985607 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf406c41-934e-4734-9e2a-1237ecea77c6 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-cf406c41-934e-4734-9e2a-1237ecea77c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:17.997443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:17.998337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:18.004158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:18.004158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:18.004158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:18.004158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:18.004158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:18.004158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:18.004158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:18.004158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:18.004158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 282/580] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:18.004158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf406c41-934e-4734-9e2a-1237ecea77c6 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:18.017898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:18.021655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adbd7c-a1bd-44ef-b1c2-738e153bba48 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:18.060031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35796914-c58c-407f-b910-22dcc01e1f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:18.060724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35796914-c58c-407f-b910-22dcc01e1f3b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:18.060911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35796914-c58c-407f-b910-22dcc01e1f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:18.061197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35796914-c58c-407f-b910-22dcc01e1f3b None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:18.061854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35796914-c58c-407f-b910-22dcc01e1f3b None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:18.062232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35796914-c58c-407f-b910-22dcc01e1f3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:18.063646 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35796914-c58c-407f-b910-22dcc01e1f3b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:18.072477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 299/581] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:18.073281 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35796914-c58c-407f-b910-22dcc01e1f3b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:18.113478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf406c41-934e-4734-9e2a-1237ecea77c6 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:18.150547 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fea33-c823-4b13-8f4a-41ac371ae9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:18.151184 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fea33-c823-4b13-8f4a-41ac371ae9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:18.151464 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fea33-c823-4b13-8f4a-41ac371ae9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:18.151770 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fea33-c823-4b13-8f4a-41ac371ae9a0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:18.159694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fea33-c823-4b13-8f4a-41ac371ae9a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:18.161148 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fea33-c823-4b13-8f4a-41ac371ae9a0 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-a57fea33-c823-4b13-8f4a-41ac371ae9a0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:18.172593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:18.173570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:18.174567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:18.174567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:18.174567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:18.174567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:18.174567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:18.174567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:18.174567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:18.174567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:18.174567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 23:15:18.174567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:18.174567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 283/582] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:15:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:18.175149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fea33-c823-4b13-8f4a-41ac371ae9a0 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:18.187454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35796914-c58c-407f-b910-22dcc01e1f3b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:18.226128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531afa9-ab89-48e9-bc51-664c9939c539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:18.226955 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531afa9-ab89-48e9-bc51-664c9939c539 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:18.227238 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531afa9-ab89-48e9-bc51-664c9939c539 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:18.227513 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531afa9-ab89-48e9-bc51-664c9939c539 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:18.228200 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531afa9-ab89-48e9-bc51-664c9939c539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:18.228500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531afa9-ab89-48e9-bc51-664c9939c539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:18.229866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531afa9-ab89-48e9-bc51-664c9939c539 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:18.239144 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:18.247240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 300/583] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:18.248054 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531afa9-ab89-48e9-bc51-664c9939c539 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:18.268868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:18.287111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fea33-c823-4b13-8f4a-41ac371ae9a0 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:18.326662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775d444-5589-4251-aee7-e1287ba32d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:18.327273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775d444-5589-4251-aee7-e1287ba32d1a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:18.327550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775d444-5589-4251-aee7-e1287ba32d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:18.327848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775d444-5589-4251-aee7-e1287ba32d1a None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:18.328986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775d444-5589-4251-aee7-e1287ba32d1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:18.329294 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775d444-5589-4251-aee7-e1287ba32d1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:18.331418 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775d444-5589-4251-aee7-e1287ba32d1a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:18.348384 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 284/584] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:18.349434 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775d444-5589-4251-aee7-e1287ba32d1a None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:18.355647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531afa9-ab89-48e9-bc51-664c9939c539 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:18.392987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e60ae-9bba-495d-87f2-9ade95cebef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:18.393931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e60ae-9bba-495d-87f2-9ade95cebef5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:18.394406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e60ae-9bba-495d-87f2-9ade95cebef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:18.394642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e60ae-9bba-495d-87f2-9ade95cebef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:18.405753 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e60ae-9bba-495d-87f2-9ade95cebef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:18.407246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e60ae-9bba-495d-87f2-9ade95cebef5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-474e60ae-9bba-495d-87f2-9ade95cebef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:18.423722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:18.424703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:18.425614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:18.425614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:18.425614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:18.425614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 301/585] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:18.426358 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e60ae-9bba-495d-87f2-9ade95cebef5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:18.457718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775d444-5589-4251-aee7-e1287ba32d1a None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:18.474920 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:18.493506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff1030-f3df-4fad-8da5-c0393ab99540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:18.494121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff1030-f3df-4fad-8da5-c0393ab99540 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:18.494402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff1030-f3df-4fad-8da5-c0393ab99540 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:18.494677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff1030-f3df-4fad-8da5-c0393ab99540 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:18.495396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff1030-f3df-4fad-8da5-c0393ab99540 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:18.495729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff1030-f3df-4fad-8da5-c0393ab99540 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:18.497165 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff1030-f3df-4fad-8da5-c0393ab99540 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:18.511028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:18.526570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 285/586] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Sep 12 23:15:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:18.527320 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff1030-f3df-4fad-8da5-c0393ab99540 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:18.549582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e60ae-9bba-495d-87f2-9ade95cebef5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:18.587027 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3786b45-7d06-4d85-8188-1f976bf7f866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:18.587808 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3786b45-7d06-4d85-8188-1f976bf7f866 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:18.588179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3786b45-7d06-4d85-8188-1f976bf7f866 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:18.588530 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3786b45-7d06-4d85-8188-1f976bf7f866 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:18.596683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3786b45-7d06-4d85-8188-1f976bf7f866 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:18.598197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3786b45-7d06-4d85-8188-1f976bf7f866 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e3786b45-7d06-4d85-8188-1f976bf7f866 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:18.609953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:18.611119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:18.612378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:18.612378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:18.612378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:18.612378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:18.612378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:18.612378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:18.612378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:18.612378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:18.612378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:15:18.612378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:18.613191 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 302/587] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:15:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:18.614037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3786b45-7d06-4d85-8188-1f976bf7f866 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:18.642203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff1030-f3df-4fad-8da5-c0393ab99540 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:18.676090 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd027d0-542d-4088-b58e-04a3d38ed0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:18.676923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd027d0-542d-4088-b58e-04a3d38ed0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:18.677202 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd027d0-542d-4088-b58e-04a3d38ed0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:18.677471 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd027d0-542d-4088-b58e-04a3d38ed0b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:18.678191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd027d0-542d-4088-b58e-04a3d38ed0b6 None admin] RBAC: Adding query filter params (role.id=5c7e1f557ae04250a01db783f3c27a5f, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=c1f6bb09fd7f4905a1693ef7cb6f088e) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:18.678496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd027d0-542d-4088-b58e-04a3d38ed0b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:18.680026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd027d0-542d-4088-b58e-04a3d38ed0b6 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:18.685160 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 286/588] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:18 2022] GET /identity/v3/role_assignments?user.id=c1f6bb09fd7f4905a1693ef7cb6f088e&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=5c7e1f557ae04250a01db783f3c27a5f => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:18.685959 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd027d0-542d-4088-b58e-04a3d38ed0b6 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:18.737742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:18.739175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3786b45-7d06-4d85-8188-1f976bf7f866 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:18.776776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:18.788677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa19c1c-3672-4262-99a4-21ba7005f0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:18.789561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa19c1c-3672-4262-99a4-21ba7005f0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:18.790492 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa19c1c-3672-4262-99a4-21ba7005f0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:18.790869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa19c1c-3672-4262-99a4-21ba7005f0c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:18.792009 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa19c1c-3672-4262-99a4-21ba7005f0c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:18.792431 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa19c1c-3672-4262-99a4-21ba7005f0c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:18.794248 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa19c1c-3672-4262-99a4-21ba7005f0c3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:18.809201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 303/589] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:18.810770 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa19c1c-3672-4262-99a4-21ba7005f0c3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:18.813333 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd027d0-542d-4088-b58e-04a3d38ed0b6 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:18.854143 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55572ba1-ed32-44cb-b68a-7a0a216e55a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:18.855103 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55572ba1-ed32-44cb-b68a-7a0a216e55a3 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:18.855596 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55572ba1-ed32-44cb-b68a-7a0a216e55a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:18.856128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55572ba1-ed32-44cb-b68a-7a0a216e55a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:18.868186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55572ba1-ed32-44cb-b68a-7a0a216e55a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:18.869662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55572ba1-ed32-44cb-b68a-7a0a216e55a3 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-55572ba1-ed32-44cb-b68a-7a0a216e55a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:18.886247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:18.887439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:18.888564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:18.888564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:18.888564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:18.890382 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 287/590] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:18.891498 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55572ba1-ed32-44cb-b68a-7a0a216e55a3 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:18.944832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa19c1c-3672-4262-99a4-21ba7005f0c3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:18.987993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b4b50-e375-4082-84d8-89126dd5ba2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:18.988953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4b50-e375-4082-84d8-89126dd5ba2c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:18.989374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4b50-e375-4082-84d8-89126dd5ba2c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:18.989737 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4b50-e375-4082-84d8-89126dd5ba2c None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:18.990563 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b4b50-e375-4082-84d8-89126dd5ba2c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:18.990964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b4b50-e375-4082-84d8-89126dd5ba2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:18.992554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b4b50-e375-4082-84d8-89126dd5ba2c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:19.014966 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 304/591] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Sep 12 23:15:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:19.016192 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b4b50-e375-4082-84d8-89126dd5ba2c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:19.032419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:19.044694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55572ba1-ed32-44cb-b68a-7a0a216e55a3 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:19.078264 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:19.092590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77937f94-8e7a-4189-91da-a7d1d662c8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:19.093559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77937f94-8e7a-4189-91da-a7d1d662c8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:19.094248 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77937f94-8e7a-4189-91da-a7d1d662c8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:19.094685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77937f94-8e7a-4189-91da-a7d1d662c8f7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:19.108314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77937f94-8e7a-4189-91da-a7d1d662c8f7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:19.110505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77937f94-8e7a-4189-91da-a7d1d662c8f7 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-77937f94-8e7a-4189-91da-a7d1d662c8f7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:19.126616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:19.127779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:19.128940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:19.128940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:19.128940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:19.128940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:19.128940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:19.128940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:19.128940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:19.128940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:19.128940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 23:15:19.128940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:19.136711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 288/592] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:19.136711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77937f94-8e7a-4189-91da-a7d1d662c8f7 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:19.144713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b4b50-e375-4082-84d8-89126dd5ba2c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:19.184157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef645c-9b57-412f-b2a0-4dc078cea0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:19.184971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef645c-9b57-412f-b2a0-4dc078cea0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:19.185385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef645c-9b57-412f-b2a0-4dc078cea0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:19.185694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef645c-9b57-412f-b2a0-4dc078cea0a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:19.187158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef645c-9b57-412f-b2a0-4dc078cea0a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:19.187158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef645c-9b57-412f-b2a0-4dc078cea0a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:19.188346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef645c-9b57-412f-b2a0-4dc078cea0a5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:19.200773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 305/593] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:19.202158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef645c-9b57-412f-b2a0-4dc078cea0a5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:19.275380 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77937f94-8e7a-4189-91da-a7d1d662c8f7 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:19.316612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b951c4b-fbe9-4c40-8de5-ca7c9a6e19d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:19.317325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b951c4b-fbe9-4c40-8de5-ca7c9a6e19d0 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:19.317697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b951c4b-fbe9-4c40-8de5-ca7c9a6e19d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:19.318064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b951c4b-fbe9-4c40-8de5-ca7c9a6e19d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:19.322720 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef645c-9b57-412f-b2a0-4dc078cea0a5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:19.331734 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b951c4b-fbe9-4c40-8de5-ca7c9a6e19d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:19.333673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b951c4b-fbe9-4c40-8de5-ca7c9a6e19d0 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:19.333989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-0b951c4b-fbe9-4c40-8de5-ca7c9a6e19d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:19.354243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:19.355460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:19.356607 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:19.356607 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:19.356607 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:19.359636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 289/594] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:19.360612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b951c4b-fbe9-4c40-8de5-ca7c9a6e19d0 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:19.366423 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:19.373113 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0ccf3-f336-4fde-bf31-7025358ee98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:19.374073 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0ccf3-f336-4fde-bf31-7025358ee98e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:19.374657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0ccf3-f336-4fde-bf31-7025358ee98e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:19.375712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0ccf3-f336-4fde-bf31-7025358ee98e None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:19.376731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0ccf3-f336-4fde-bf31-7025358ee98e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:19.377725 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0ccf3-f336-4fde-bf31-7025358ee98e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:19.380351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0ccf3-f336-4fde-bf31-7025358ee98e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:19.407470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 306/595] 10.222.0.27 () {56 vars in 1139 bytes} [Mon Sep 12 23:15:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:19.408675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0ccf3-f336-4fde-bf31-7025358ee98e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:19.477583 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b951c4b-fbe9-4c40-8de5-ca7c9a6e19d0 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:19.517464 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c65321-e8cb-413e-933d-94934beb493a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:19.519016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c65321-e8cb-413e-933d-94934beb493a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:19.519614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c65321-e8cb-413e-933d-94934beb493a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:19.520062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c65321-e8cb-413e-933d-94934beb493a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:19.529545 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c65321-e8cb-413e-933d-94934beb493a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:19.532161 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c65321-e8cb-413e-933d-94934beb493a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:19.539885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0ccf3-f336-4fde-bf31-7025358ee98e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-f5c65321-e8cb-413e-933d-94934beb493a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:19.547475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:19.548566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:19.549647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:19.549647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:19.549647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:19.549647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:19.549647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:19.549647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:19.549647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:19.549647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:19.552145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 290/596] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:19.553111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c65321-e8cb-413e-933d-94934beb493a None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:19.592388 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf90f14-95f1-46e6-b1a3-f3cd4f3f5a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:19.593039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf90f14-95f1-46e6-b1a3-f3cd4f3f5a37 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:19.593541 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf90f14-95f1-46e6-b1a3-f3cd4f3f5a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:19.593998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf90f14-95f1-46e6-b1a3-f3cd4f3f5a37 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:19.594749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf90f14-95f1-46e6-b1a3-f3cd4f3f5a37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:19.595140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf90f14-95f1-46e6-b1a3-f3cd4f3f5a37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:19.596979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf90f14-95f1-46e6-b1a3-f3cd4f3f5a37 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:19.616821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 307/597] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:19.617627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf90f14-95f1-46e6-b1a3-f3cd4f3f5a37 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:19.637620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:19.648069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:19.683294 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c65321-e8cb-413e-933d-94934beb493a None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:19.720737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bff28-851b-40ad-b58a-dddec2382a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:19.721765 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bff28-851b-40ad-b58a-dddec2382a7c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:19.721983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf90f14-95f1-46e6-b1a3-f3cd4f3f5a37 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:19.722467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bff28-851b-40ad-b58a-dddec2382a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:19.722970 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bff28-851b-40ad-b58a-dddec2382a7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:19.734772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1bff28-851b-40ad-b58a-dddec2382a7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:19.736675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1bff28-851b-40ad-b58a-dddec2382a7c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-5e1bff28-851b-40ad-b58a-dddec2382a7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:19.752856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:19.753808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:19.754689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:19.754689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:19.754689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:19.756187 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 291/598] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:19.757112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bff28-851b-40ad-b58a-dddec2382a7c None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:19.759202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fb8e0-673d-4dee-ae72-10bdb2a2d028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:19.759996 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fb8e0-673d-4dee-ae72-10bdb2a2d028 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:19.760420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fb8e0-673d-4dee-ae72-10bdb2a2d028 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:19.761024 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fb8e0-673d-4dee-ae72-10bdb2a2d028 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:19.762380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771fb8e0-673d-4dee-ae72-10bdb2a2d028 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:19.762380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771fb8e0-673d-4dee-ae72-10bdb2a2d028 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:19.763784 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771fb8e0-673d-4dee-ae72-10bdb2a2d028 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:19.784473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 308/599] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:19.785411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fb8e0-673d-4dee-ae72-10bdb2a2d028 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:19.872161 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bff28-851b-40ad-b58a-dddec2382a7c None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:19.877091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:19.889936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:19.891135 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fb8e0-673d-4dee-ae72-10bdb2a2d028 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:19.915986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308cb67-2dd5-47e0-bc6d-bfaf98c6594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:19.916894 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308cb67-2dd5-47e0-bc6d-bfaf98c6594c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:19.917335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308cb67-2dd5-47e0-bc6d-bfaf98c6594c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:19.917755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308cb67-2dd5-47e0-bc6d-bfaf98c6594c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:19.925128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308cb67-2dd5-47e0-bc6d-bfaf98c6594c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:19.926574 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308cb67-2dd5-47e0-bc6d-bfaf98c6594c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:19.933085 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ebe5c-9d11-43d7-a6c5-00016d6b7899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:19.933768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ebe5c-9d11-43d7-a6c5-00016d6b7899 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:19.934056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ebe5c-9d11-43d7-a6c5-00016d6b7899 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:19.934330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ebe5c-9d11-43d7-a6c5-00016d6b7899 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:19.934974 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ebe5c-9d11-43d7-a6c5-00016d6b7899 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:19.935280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ebe5c-9d11-43d7-a6c5-00016d6b7899 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:19.936705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ebe5c-9d11-43d7-a6c5-00016d6b7899 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-f308cb67-2dd5-47e0-bc6d-bfaf98c6594c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:19.941541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:19.942407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:19.943295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:19.943295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:19.943295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:19.943295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:19.943295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:19.943295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:19.943295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:19.943295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:19.943295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 292/600] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:19.943826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308cb67-2dd5-47e0-bc6d-bfaf98c6594c None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:19.951134 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 309/601] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:19.952418 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0ebca-eb7d-419a-9b11-939f11c3ccca None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:19.952706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0ebca-eb7d-419a-9b11-939f11c3ccca None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:19.952983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0ebca-eb7d-419a-9b11-939f11c3ccca None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:19.954258 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 310/602] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:19.954958 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0ebca-eb7d-419a-9b11-939f11c3ccca None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:20.058604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308cb67-2dd5-47e0-bc6d-bfaf98c6594c None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:20.063314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0ebca-eb7d-419a-9b11-939f11c3ccca None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:20.099458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f9094-7244-4fc0-a9da-ba1eae9f18ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:20.100099 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f9094-7244-4fc0-a9da-ba1eae9f18ca None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:20.100234 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141dd2ac-c0d8-41d6-bab6-fe32369158b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:20.100373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f9094-7244-4fc0-a9da-ba1eae9f18ca None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:20.100621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f9094-7244-4fc0-a9da-ba1eae9f18ca None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:20.101117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dd2ac-c0d8-41d6-bab6-fe32369158b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:20.101312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f9094-7244-4fc0-a9da-ba1eae9f18ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:20.101589 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dd2ac-c0d8-41d6-bab6-fe32369158b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:20.101682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f9094-7244-4fc0-a9da-ba1eae9f18ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:20.102018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dd2ac-c0d8-41d6-bab6-fe32369158b9 None admin] PATH_INFO: `/v3/projects/32bbb936501846e8914acf07890c2891/users/d61407406ef242a78030f79532fe2247/roles/5c7e1f557ae04250a01db783f3c27a5f` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:20.103068 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f9094-7244-4fc0-a9da-ba1eae9f18ca None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:20.116820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141dd2ac-c0d8-41d6-bab6-fe32369158b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=32bbb936501846e8914acf07890c2891, user_id=d61407406ef242a78030f79532fe2247, role_id=5c7e1f557ae04250a01db783f3c27a5f)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:20.121848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 311/603] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:20.121974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141dd2ac-c0d8-41d6-bab6-fe32369158b9 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:20.122050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f9094-7244-4fc0-a9da-ba1eae9f18ca None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:20.132556 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:20.136747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 293/604] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:15:19 2022] PUT /identity/v3/projects/32bbb936501846e8914acf07890c2891/users/d61407406ef242a78030f79532fe2247/roles/5c7e1f557ae04250a01db783f3c27a5f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:15:20.138313 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ab344-69b4-46a6-90b0-a179fee5256e None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:20.138600 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ab344-69b4-46a6-90b0-a179fee5256e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:20.138883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ab344-69b4-46a6-90b0-a179fee5256e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:20.166628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-4c6ab344-69b4-46a6-90b0-a179fee5256e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:20.167950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4c6ab344-69b4-46a6-90b0-a179fee5256e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:20.192876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:20.264813 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 294/605] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:20.265689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ab344-69b4-46a6-90b0-a179fee5256e None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:20.266277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f9094-7244-4fc0-a9da-ba1eae9f18ca None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:20.326954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbe764-d16d-4204-8289-947bef043c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:20.327740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbe764-d16d-4204-8289-947bef043c97 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:20.328085 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbe764-d16d-4204-8289-947bef043c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:20.328431 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbe764-d16d-4204-8289-947bef043c97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:20.340492 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbbe764-d16d-4204-8289-947bef043c97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:20.342012 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbbe764-d16d-4204-8289-947bef043c97 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-fcbbe764-d16d-4204-8289-947bef043c97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:20.358162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:20.359299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:20.363384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:20.363384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:20.363384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:20.363384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:20.363384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:20.363384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:20.363384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:20.363384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:20.363384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 312/606] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:20.366143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbe764-d16d-4204-8289-947bef043c97 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:20.433681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ab344-69b4-46a6-90b0-a179fee5256e None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:20.447533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:20.491559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fa97b-670e-4012-a6f6-c2eb9fb9d397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:20.493054 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fa97b-670e-4012-a6f6-c2eb9fb9d397 None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:20.494184 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fa97b-670e-4012-a6f6-c2eb9fb9d397 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:20.495213 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fa97b-670e-4012-a6f6-c2eb9fb9d397 None admin] PATH_INFO: `/v3/projects/32bbb936501846e8914acf07890c2891/users/c5b8828077594349a93c7a842c0f15da/roles/5c7e1f557ae04250a01db783f3c27a5f` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:20.502047 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbe764-d16d-4204-8289-947bef043c97 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:20.527014 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fa97b-670e-4012-a6f6-c2eb9fb9d397 None admin] RBAC: Authorizing `identity:create_grant(project_id=32bbb936501846e8914acf07890c2891, user_id=c5b8828077594349a93c7a842c0f15da, role_id=5c7e1f557ae04250a01db783f3c27a5f)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:20.528764 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fa97b-670e-4012-a6f6-c2eb9fb9d397 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:20.549574 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 295/607] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:15:20 2022] PUT /identity/v3/projects/32bbb936501846e8914acf07890c2891/users/c5b8828077594349a93c7a842c0f15da/roles/5c7e1f557ae04250a01db783f3c27a5f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:15:20.572288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311055f-e3b0-452a-977f-7447a563d579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:20.573080 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311055f-e3b0-452a-977f-7447a563d579 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:20.573464 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311055f-e3b0-452a-977f-7447a563d579 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:20.573848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311055f-e3b0-452a-977f-7447a563d579 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:20.574649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311055f-e3b0-452a-977f-7447a563d579 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:20.575058 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311055f-e3b0-452a-977f-7447a563d579 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:20.576623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311055f-e3b0-452a-977f-7447a563d579 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:20.597647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 313/608] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:20.604465 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fa97b-670e-4012-a6f6-c2eb9fb9d397 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:20.612855 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:20.627687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311055f-e3b0-452a-977f-7447a563d579 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:20.762940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311055f-e3b0-452a-977f-7447a563d579 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:20.799462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fa97b-670e-4012-a6f6-c2eb9fb9d397 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:20.804069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:20.809697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb76692-17e1-44ed-8ed4-2709f5dabc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:20.810420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb76692-17e1-44ed-8ed4-2709f5dabc49 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:20.810818 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb76692-17e1-44ed-8ed4-2709f5dabc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:20.811236 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb76692-17e1-44ed-8ed4-2709f5dabc49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:20.823427 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb76692-17e1-44ed-8ed4-2709f5dabc49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4eb76692-17e1-44ed-8ed4-2709f5dabc49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:20.829330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:20.830283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:20.831132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:20.831132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:20.831132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:20.831132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:20.831132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:20.831132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:20.831132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:20.831132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:20.831132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:20.831132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:20.831132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:20.831132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:20.835498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 314/609] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:20.837293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb76692-17e1-44ed-8ed4-2709f5dabc49 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:20.841196 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32083b7-ca59-4937-8ec5-a392ce1c19f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:20.841895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32083b7-ca59-4937-8ec5-a392ce1c19f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:20.842119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32083b7-ca59-4937-8ec5-a392ce1c19f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:20.842390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32083b7-ca59-4937-8ec5-a392ce1c19f8 None admin] PATH_INFO: `/v3/projects/32bbb936501846e8914acf07890c2891/users/bc6e63e9fc3447d5a894a777a98fe773/roles/5c7e1f557ae04250a01db783f3c27a5f` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:20.859523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32083b7-ca59-4937-8ec5-a392ce1c19f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=32bbb936501846e8914acf07890c2891, user_id=bc6e63e9fc3447d5a894a777a98fe773, role_id=5c7e1f557ae04250a01db783f3c27a5f)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:20.861049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32083b7-ca59-4937-8ec5-a392ce1c19f8 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:20.880979 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 296/610] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:15:20 2022] PUT /identity/v3/projects/32bbb936501846e8914acf07890c2891/users/bc6e63e9fc3447d5a894a777a98fe773/roles/5c7e1f557ae04250a01db783f3c27a5f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:15:20.984065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce598c6b-5796-495f-8b31-ebdc18d54881 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:20.984556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce598c6b-5796-495f-8b31-ebdc18d54881 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:20.985029 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce598c6b-5796-495f-8b31-ebdc18d54881 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:20.987148 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 297/611] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:21.000108 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68544505-18e7-456b-b2e5-321d17d16dc1 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:21.000460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68544505-18e7-456b-b2e5-321d17d16dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:21.001027 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68544505-18e7-456b-b2e5-321d17d16dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:21.018796 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb76692-17e1-44ed-8ed4-2709f5dabc49 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:21.030206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-68544505-18e7-456b-b2e5-321d17d16dc1 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:21.031469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-68544505-18e7-456b-b2e5-321d17d16dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:21.094339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710b7cf-301d-49ee-abf7-afb7e6464fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:21.094952 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710b7cf-301d-49ee-abf7-afb7e6464fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:21.095231 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710b7cf-301d-49ee-abf7-afb7e6464fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:21.095499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710b7cf-301d-49ee-abf7-afb7e6464fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:21.096184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710b7cf-301d-49ee-abf7-afb7e6464fa6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:21.096486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710b7cf-301d-49ee-abf7-afb7e6464fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:21.097856 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710b7cf-301d-49ee-abf7-afb7e6464fa6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:21.109303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 315/612] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:21.114924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710b7cf-301d-49ee-abf7-afb7e6464fa6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:21.124529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:21.146885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 298/613] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:21.192639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:21.263952 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710b7cf-301d-49ee-abf7-afb7e6464fa6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:21.332928 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a43968-b66c-43ad-8f7e-cb713933ee73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:21.333536 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a43968-b66c-43ad-8f7e-cb713933ee73 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:21.333822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a43968-b66c-43ad-8f7e-cb713933ee73 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:21.334096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a43968-b66c-43ad-8f7e-cb713933ee73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:21.351103 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a43968-b66c-43ad-8f7e-cb713933ee73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:21.353138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a43968-b66c-43ad-8f7e-cb713933ee73 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-00a43968-b66c-43ad-8f7e-cb713933ee73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:21.379481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:21.380501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:21.381396 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:21.381396 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:21.381396 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:21.383765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 316/614] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:15:21.388962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68544505-18e7-456b-b2e5-321d17d16dc1 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:21.483347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a43968-b66c-43ad-8f7e-cb713933ee73 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:21.589964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68544505-18e7-456b-b2e5-321d17d16dc1 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:21.669205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34afdc9-6874-493d-9694-0ea90590ce20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:21.669835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34afdc9-6874-493d-9694-0ea90590ce20 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:21.670116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34afdc9-6874-493d-9694-0ea90590ce20 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:21.670388 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34afdc9-6874-493d-9694-0ea90590ce20 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:21.671031 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34afdc9-6874-493d-9694-0ea90590ce20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:21.671334 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34afdc9-6874-493d-9694-0ea90590ce20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:21.672661 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a43968-b66c-43ad-8f7e-cb713933ee73 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:21.672775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34afdc9-6874-493d-9694-0ea90590ce20 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:21.674391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:21.699248 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 299/615] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:21.706219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34afdc9-6874-493d-9694-0ea90590ce20 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:21.728316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0ba9a-3ae2-4c6a-838a-0aafbb337ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:21.729062 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0ba9a-3ae2-4c6a-838a-0aafbb337ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:21.729340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0ba9a-3ae2-4c6a-838a-0aafbb337ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:21.729742 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0ba9a-3ae2-4c6a-838a-0aafbb337ad3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:21.755184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0ba9a-3ae2-4c6a-838a-0aafbb337ad3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:21.761342 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-0ca0ba9a-3ae2-4c6a-838a-0aafbb337ad3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:21.764709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:21.765715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:21.766550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:21.766550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:21.766550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:21.766550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:21.766550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:21.766550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:21.766550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:21.766550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:21.766550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:21.766550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:21.766550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:21.766550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:21.769022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 317/616] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:15:21.777205 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0ba9a-3ae2-4c6a-838a-0aafbb337ad3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:21.873859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34afdc9-6874-493d-9694-0ea90590ce20 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:21.889546 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0ba9a-3ae2-4c6a-838a-0aafbb337ad3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:21.947986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a919a82-b0cc-42b8-b5bc-84d57180f633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:21.949758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a919a82-b0cc-42b8-b5bc-84d57180f633 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:21.950963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a919a82-b0cc-42b8-b5bc-84d57180f633 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:21.951849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a919a82-b0cc-42b8-b5bc-84d57180f633 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:21.958148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0215-7f5d-4d45-9230-ed9ef1c82e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:21.959041 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0215-7f5d-4d45-9230-ed9ef1c82e07 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:21.959554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0215-7f5d-4d45-9230-ed9ef1c82e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:21.964297 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0215-7f5d-4d45-9230-ed9ef1c82e07 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:21.965189 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e0215-7f5d-4d45-9230-ed9ef1c82e07 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:21.965729 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e0215-7f5d-4d45-9230-ed9ef1c82e07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:21.967309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e0215-7f5d-4d45-9230-ed9ef1c82e07 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:21.970148 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a919a82-b0cc-42b8-b5bc-84d57180f633 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:21.972870 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a919a82-b0cc-42b8-b5bc-84d57180f633 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:21.984999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 318/617] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:21.987315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0215-7f5d-4d45-9230-ed9ef1c82e07 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-7a919a82-b0cc-42b8-b5bc-84d57180f633 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:22.001492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:22.002428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:22.003374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:22.003374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:22.003374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:22.003374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:22.003374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:22.003374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:22.003374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:22.003374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:22.003374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:15:22.003374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:22.006003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 300/618] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Sep 12 23:15:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:22.011033 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a919a82-b0cc-42b8-b5bc-84d57180f633 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:22.047961 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:22.151969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a919a82-b0cc-42b8-b5bc-84d57180f633 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:22.161253 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:22.195185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0215-7f5d-4d45-9230-ed9ef1c82e07 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:22.229788 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a87ca7-440f-43e3-baf8-00d8523bb39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:22.230414 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a87ca7-440f-43e3-baf8-00d8523bb39e None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:22.230692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a87ca7-440f-43e3-baf8-00d8523bb39e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:22.230960 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a87ca7-440f-43e3-baf8-00d8523bb39e None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:22.231716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a87ca7-440f-43e3-baf8-00d8523bb39e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:22.232214 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a87ca7-440f-43e3-baf8-00d8523bb39e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:22.233307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657c05d-6a9c-4495-90fc-e6256c86c35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:22.233662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a87ca7-440f-43e3-baf8-00d8523bb39e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:22.233989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657c05d-6a9c-4495-90fc-e6256c86c35b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:22.234226 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657c05d-6a9c-4495-90fc-e6256c86c35b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:22.234501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657c05d-6a9c-4495-90fc-e6256c86c35b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:22.246411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e657c05d-6a9c-4495-90fc-e6256c86c35b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:22.247888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e657c05d-6a9c-4495-90fc-e6256c86c35b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e657c05d-6a9c-4495-90fc-e6256c86c35b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:22.264439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:22.265403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:22.266296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:22.266296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:22.266296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:22.266296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 319/619] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:22.271973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657c05d-6a9c-4495-90fc-e6256c86c35b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:22.274399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 301/620] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Sep 12 23:15:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:22.279595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a87ca7-440f-43e3-baf8-00d8523bb39e None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:22.357435 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:22.384869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657c05d-6a9c-4495-90fc-e6256c86c35b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:22.423291 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18641d5-d999-4c09-8d2f-2d5cd1b32bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:22.424125 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18641d5-d999-4c09-8d2f-2d5cd1b32bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:22.424503 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18641d5-d999-4c09-8d2f-2d5cd1b32bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:22.424876 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18641d5-d999-4c09-8d2f-2d5cd1b32bb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:22.425620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18641d5-d999-4c09-8d2f-2d5cd1b32bb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:22.426019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18641d5-d999-4c09-8d2f-2d5cd1b32bb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:22.427482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18641d5-d999-4c09-8d2f-2d5cd1b32bb6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:22.441396 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 320/621] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:22.448575 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18641d5-d999-4c09-8d2f-2d5cd1b32bb6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:22.461461 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a87ca7-440f-43e3-baf8-00d8523bb39e None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:22.498776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba51afe-30db-479e-afb2-c57b4c03ccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:22.499549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba51afe-30db-479e-afb2-c57b4c03ccd6 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:22.499958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba51afe-30db-479e-afb2-c57b4c03ccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:22.500337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba51afe-30db-479e-afb2-c57b4c03ccd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:22.565656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:22.573698 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba51afe-30db-479e-afb2-c57b4c03ccd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:22.575872 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba51afe-30db-479e-afb2-c57b4c03ccd6 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:22.673265 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18641d5-d999-4c09-8d2f-2d5cd1b32bb6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:22.727419 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf719f-1b62-4989-90ad-4dd9ebb84b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:22.727992 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf719f-1b62-4989-90ad-4dd9ebb84b4a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:22.728270 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf719f-1b62-4989-90ad-4dd9ebb84b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:22.728543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf719f-1b62-4989-90ad-4dd9ebb84b4a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:22.731904 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:22.736898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf719f-1b62-4989-90ad-4dd9ebb84b4a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:22.738362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf719f-1b62-4989-90ad-4dd9ebb84b4a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:22.738888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 302/622] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:15:22.742917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d759f-5245-46a2-a55e-5e7802f7b91d None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:22.743486 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d759f-5245-46a2-a55e-5e7802f7b91d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:22.743998 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d759f-5245-46a2-a55e-5e7802f7b91d None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:22.745776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 303/623] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:22.746831 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d759f-5245-46a2-a55e-5e7802f7b91d None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-dacf719f-1b62-4989-90ad-4dd9ebb84b4a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:22.753250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:22.754221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:22.755170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:22.755170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:22.755170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:22.755170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:22.755170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:22.755170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:22.755170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:22.755170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:22.755170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:15:22.755170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:22.755170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 321/624] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:15:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:22.755803 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d79c25-e63d-4d23-8121-da14ceeaf16d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:22.756440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d79c25-e63d-4d23-8121-da14ceeaf16d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:22.756440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d79c25-e63d-4d23-8121-da14ceeaf16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:22.775271 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-01d79c25-e63d-4d23-8121-da14ceeaf16d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:22.776477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-01d79c25-e63d-4d23-8121-da14ceeaf16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:22.835772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:22.853386 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 322/625] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:22.854148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d79c25-e63d-4d23-8121-da14ceeaf16d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:22.890845 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d759f-5245-46a2-a55e-5e7802f7b91d None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:22.937379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3b2be-a1af-414e-a565-f293fb5f6582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:22.938208 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3b2be-a1af-414e-a565-f293fb5f6582 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:22.939338 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3b2be-a1af-414e-a565-f293fb5f6582 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:22.940459 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3b2be-a1af-414e-a565-f293fb5f6582 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:22.963519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3b2be-a1af-414e-a565-f293fb5f6582 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:22.966105 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3b2be-a1af-414e-a565-f293fb5f6582 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-ccc3b2be-a1af-414e-a565-f293fb5f6582 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:23.003742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:23.004728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:23.005631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:23.005631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:23.005631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:23.005631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 304/626] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:23.010644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3b2be-a1af-414e-a565-f293fb5f6582 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:23.020861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d79c25-e63d-4d23-8121-da14ceeaf16d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:23.051541 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:23.062974 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55c07c-ff31-487e-bde9-88e8653a0ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:23.064644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55c07c-ff31-487e-bde9-88e8653a0ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:23.065196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55c07c-ff31-487e-bde9-88e8653a0ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:23.065673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55c07c-ff31-487e-bde9-88e8653a0ca3 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:23.066663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55c07c-ff31-487e-bde9-88e8653a0ca3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:23.067216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55c07c-ff31-487e-bde9-88e8653a0ca3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:23.069451 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55c07c-ff31-487e-bde9-88e8653a0ca3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:23.092892 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 323/627] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Sep 12 23:15:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:23.099844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55c07c-ff31-487e-bde9-88e8653a0ca3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:23.188444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3b2be-a1af-414e-a565-f293fb5f6582 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:23.208064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:23.229395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54cfac-eb19-49b1-8296-de7b04c41ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:23.230094 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54cfac-eb19-49b1-8296-de7b04c41ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:23.230448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54cfac-eb19-49b1-8296-de7b04c41ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:23.230792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54cfac-eb19-49b1-8296-de7b04c41ef4 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:23.231508 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54cfac-eb19-49b1-8296-de7b04c41ef4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:23.231910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54cfac-eb19-49b1-8296-de7b04c41ef4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:23.233630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54cfac-eb19-49b1-8296-de7b04c41ef4 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:23.246283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 305/628] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:23.247657 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918ee28-fba9-469a-9274-5699a0dc2aef None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:23.247947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918ee28-fba9-469a-9274-5699a0dc2aef None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:23.248224 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918ee28-fba9-469a-9274-5699a0dc2aef None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:23.249544 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 306/629] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:23.252593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918ee28-fba9-469a-9274-5699a0dc2aef None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:23.264281 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55c07c-ff31-487e-bde9-88e8653a0ca3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:23.308182 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b755d-06cf-4cdd-a3c1-26ed40e9a3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:23.308823 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b755d-06cf-4cdd-a3c1-26ed40e9a3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:23.309138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b755d-06cf-4cdd-a3c1-26ed40e9a3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:23.309412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b755d-06cf-4cdd-a3c1-26ed40e9a3c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:23.320019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b755d-06cf-4cdd-a3c1-26ed40e9a3c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:23.323082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b755d-06cf-4cdd-a3c1-26ed40e9a3c3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a41b755d-06cf-4cdd-a3c1-26ed40e9a3c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:23.339783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:23.340774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:23.341674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:23.341674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:23.341674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:23.341674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 324/630] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:23.342487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b755d-06cf-4cdd-a3c1-26ed40e9a3c3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:23.374053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918ee28-fba9-469a-9274-5699a0dc2aef None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:23.381989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:23.413799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3529967-3e98-462c-9a80-ab0f1a5342e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:23.414518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3529967-3e98-462c-9a80-ab0f1a5342e4 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:23.414800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3529967-3e98-462c-9a80-ab0f1a5342e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:23.415076 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3529967-3e98-462c-9a80-ab0f1a5342e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:23.426401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3529967-3e98-462c-9a80-ab0f1a5342e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-f3529967-3e98-462c-9a80-ab0f1a5342e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:23.431852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:23.432760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:23.433630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:23.433630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:23.433630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:23.433630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:23.433630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:23.433630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:23.433630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:23.433630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:23.433630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:23.433630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:23.433630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:23.433630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:23.433630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 307/631] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:23.434343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14f0a0-0d21-40de-a1b8-54dc4de7dde5 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:23.435229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14f0a0-0d21-40de-a1b8-54dc4de7dde5 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:23.435229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14f0a0-0d21-40de-a1b8-54dc4de7dde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:23.452863 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-0a14f0a0-0d21-40de-a1b8-54dc4de7dde5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:23.454043 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-0a14f0a0-0d21-40de-a1b8-54dc4de7dde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:23.461569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b755d-06cf-4cdd-a3c1-26ed40e9a3c3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:23.481537 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:23.499634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c02080-3e1f-4a79-9e45-563f30fe4415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:23.500325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c02080-3e1f-4a79-9e45-563f30fe4415 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:23.500604 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c02080-3e1f-4a79-9e45-563f30fe4415 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:23.500880 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c02080-3e1f-4a79-9e45-563f30fe4415 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:23.508101 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c02080-3e1f-4a79-9e45-563f30fe4415 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:23.509537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c02080-3e1f-4a79-9e45-563f30fe4415 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a5c02080-3e1f-4a79-9e45-563f30fe4415 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:23.520679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:23.521565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:23.522497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:23.522497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:23.522497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:23.522497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:23.522497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:23.522497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:23.522497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:23.522497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:23.522497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 325/632] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:23.522497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c02080-3e1f-4a79-9e45-563f30fe4415 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:23.530198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 308/633] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:23.531038 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14f0a0-0d21-40de-a1b8-54dc4de7dde5 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:23.622548 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:23.629863 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c02080-3e1f-4a79-9e45-563f30fe4415 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:23.633919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14f0a0-0d21-40de-a1b8-54dc4de7dde5 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:23.667606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d3463-cee9-411b-956e-eec0585b6898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:23.668252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d3463-cee9-411b-956e-eec0585b6898 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:23.668536 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d3463-cee9-411b-956e-eec0585b6898 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:23.668816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d3463-cee9-411b-956e-eec0585b6898 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:23.669459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d3463-cee9-411b-956e-eec0585b6898 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:23.669770 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d3463-cee9-411b-956e-eec0585b6898 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:23.671164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d3463-cee9-411b-956e-eec0585b6898 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:23.674896 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ada295-f2df-4f26-be67-5241b830fcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:23.675511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ada295-f2df-4f26-be67-5241b830fcbe None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:23.675839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ada295-f2df-4f26-be67-5241b830fcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:23.676120 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ada295-f2df-4f26-be67-5241b830fcbe None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:23.676782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ada295-f2df-4f26-be67-5241b830fcbe None admin] RBAC: Adding query filter params (name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:23.677116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ada295-f2df-4f26-be67-5241b830fcbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:23.678477 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ada295-f2df-4f26-be67-5241b830fcbe None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:23.694678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 326/634] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:23.694678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9f9fb-5081-41dd-970c-090f7298593b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:23.694678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9f9fb-5081-41dd-970c-090f7298593b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:23.695016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 309/635] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:23.695126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9f9fb-5081-41dd-970c-090f7298593b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:23.695222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ada295-f2df-4f26-be67-5241b830fcbe None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:23.695340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 327/636] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:23.695340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9f9fb-5081-41dd-970c-090f7298593b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:23.727521 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:23.805764 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ada295-f2df-4f26-be67-5241b830fcbe None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:23.812835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9f9fb-5081-41dd-970c-090f7298593b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:23.839674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386937a8-c35c-4f5a-8765-1b9e39abaf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:23.840277 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386937a8-c35c-4f5a-8765-1b9e39abaf1a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:23.840556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386937a8-c35c-4f5a-8765-1b9e39abaf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:23.840828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386937a8-c35c-4f5a-8765-1b9e39abaf1a None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:23.841511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386937a8-c35c-4f5a-8765-1b9e39abaf1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:23.841818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386937a8-c35c-4f5a-8765-1b9e39abaf1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:23.843251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386937a8-c35c-4f5a-8765-1b9e39abaf1a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:23.846216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04fa92-6df8-4363-a842-06fff97b2459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:23.846832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04fa92-6df8-4363-a842-06fff97b2459 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:23.847105 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04fa92-6df8-4363-a842-06fff97b2459 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:23.847372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04fa92-6df8-4363-a842-06fff97b2459 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:23.854325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04fa92-6df8-4363-a842-06fff97b2459 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:23.855272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 310/637] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:23.855774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04fa92-6df8-4363-a842-06fff97b2459 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:23.856546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154bf33-700b-43c1-8737-e64f6208c59a None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:23.856840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154bf33-700b-43c1-8737-e64f6208c59a None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:23.857126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154bf33-700b-43c1-8737-e64f6208c59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-2b04fa92-6df8-4363-a842-06fff97b2459 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:23.870322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:23.871286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:23.872771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:23.872771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:23.872771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:23.872771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:23.872771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:23.872771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:23.872771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:23.872771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:23.872771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 328/638] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:23.872771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04fa92-6df8-4363-a842-06fff97b2459 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:23.876838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-5154bf33-700b-43c1-8737-e64f6208c59a None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:23.878013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5154bf33-700b-43c1-8737-e64f6208c59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:23.892521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:23.948999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 311/639] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:23.949834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154bf33-700b-43c1-8737-e64f6208c59a None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:23.976062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:23.980771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04fa92-6df8-4363-a842-06fff97b2459 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:24.016234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2ed2a-02d7-4084-b718-6d7dabeb22e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:24.017157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2ed2a-02d7-4084-b718-6d7dabeb22e9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:24.017442 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2ed2a-02d7-4084-b718-6d7dabeb22e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:24.017714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2ed2a-02d7-4084-b718-6d7dabeb22e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:24.028562 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2ed2a-02d7-4084-b718-6d7dabeb22e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:24.029998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2ed2a-02d7-4084-b718-6d7dabeb22e9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-71a2ed2a-02d7-4084-b718-6d7dabeb22e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:24.045712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:24.046838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:24.047774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:24.047774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:24.047774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:24.047774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 329/640] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:24.047774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2ed2a-02d7-4084-b718-6d7dabeb22e9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:24.072618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154bf33-700b-43c1-8737-e64f6208c59a None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:24.106633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8480f0b-cb03-4709-ac69-fe5d646d9f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:24.107262 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8480f0b-cb03-4709-ac69-fe5d646d9f5d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:24.107786 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8480f0b-cb03-4709-ac69-fe5d646d9f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:24.107926 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8480f0b-cb03-4709-ac69-fe5d646d9f5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:24.117882 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8480f0b-cb03-4709-ac69-fe5d646d9f5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:24.118867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-c8480f0b-cb03-4709-ac69-fe5d646d9f5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:24.122858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:24.123929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:24.124792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:24.124792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:24.124792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:24.124792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:24.124792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:24.124792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:24.124792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:24.124792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:24.124792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:24.124792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:24.124792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:24.124792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:24.124792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 312/641] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:24.124792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8480f0b-cb03-4709-ac69-fe5d646d9f5d None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:24.156081 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2ed2a-02d7-4084-b718-6d7dabeb22e9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:24.193806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540327eb-aba8-4d47-9635-bbe7037b1ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:24.194404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540327eb-aba8-4d47-9635-bbe7037b1ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:24.194683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540327eb-aba8-4d47-9635-bbe7037b1ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:24.194952 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540327eb-aba8-4d47-9635-bbe7037b1ee7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:24.196325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540327eb-aba8-4d47-9635-bbe7037b1ee7 None admin] RBAC: Adding query filter params (role.id=5c7e1f557ae04250a01db783f3c27a5f, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=f5fa28ffdafd435cb2d93bd453add3cb) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:24.196665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540327eb-aba8-4d47-9635-bbe7037b1ee7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:24.198073 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540327eb-aba8-4d47-9635-bbe7037b1ee7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:24.211608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 330/642] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:24 2022] GET /identity/v3/role_assignments?user.id=f5fa28ffdafd435cb2d93bd453add3cb&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=5c7e1f557ae04250a01db783f3c27a5f => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:24.211608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540327eb-aba8-4d47-9635-bbe7037b1ee7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:24.215240 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:24.240747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8480f0b-cb03-4709-ac69-fe5d646d9f5d None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:24.281507 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e827a-0062-42f7-ada1-4ccff65236e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:24.282232 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e827a-0062-42f7-ada1-4ccff65236e7 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:24.282521 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e827a-0062-42f7-ada1-4ccff65236e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:24.283110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e827a-0062-42f7-ada1-4ccff65236e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:24.283792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e827a-0062-42f7-ada1-4ccff65236e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:24.284100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e827a-0062-42f7-ada1-4ccff65236e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:24.285576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e827a-0062-42f7-ada1-4ccff65236e7 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:24.298536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 313/643] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:24.299780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e827a-0062-42f7-ada1-4ccff65236e7 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:24.339074 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540327eb-aba8-4d47-9635-bbe7037b1ee7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:24.373621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:24.380351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf616a66-dc15-4f79-a898-5be1cf394098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:24.380968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf616a66-dc15-4f79-a898-5be1cf394098 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:24.381249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf616a66-dc15-4f79-a898-5be1cf394098 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:24.381799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf616a66-dc15-4f79-a898-5be1cf394098 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:24.382869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf616a66-dc15-4f79-a898-5be1cf394098 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:24.383253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf616a66-dc15-4f79-a898-5be1cf394098 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:24.384662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf616a66-dc15-4f79-a898-5be1cf394098 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:24.401078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 331/644] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:24.401844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf616a66-dc15-4f79-a898-5be1cf394098 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:24.411114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e827a-0062-42f7-ada1-4ccff65236e7 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:24.452434 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafeb0bd-e666-44c5-9d73-c3940e50f0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:24.453053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafeb0bd-e666-44c5-9d73-c3940e50f0ec None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:24.453335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafeb0bd-e666-44c5-9d73-c3940e50f0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:24.453607 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafeb0bd-e666-44c5-9d73-c3940e50f0ec None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:24.454270 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafeb0bd-e666-44c5-9d73-c3940e50f0ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:24.454577 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafeb0bd-e666-44c5-9d73-c3940e50f0ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:24.456039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafeb0bd-e666-44c5-9d73-c3940e50f0ec None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:24.458785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:24.466416 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 314/645] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:24.467224 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafeb0bd-e666-44c5-9d73-c3940e50f0ec None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:24.518029 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf616a66-dc15-4f79-a898-5be1cf394098 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:24.558995 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52608a9-aa4f-4353-9d93-e9e00202f79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:24.559909 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52608a9-aa4f-4353-9d93-e9e00202f79c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:24.560190 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52608a9-aa4f-4353-9d93-e9e00202f79c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:24.560458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52608a9-aa4f-4353-9d93-e9e00202f79c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:24.567530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafeb0bd-e666-44c5-9d73-c3940e50f0ec None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:24.572758 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52608a9-aa4f-4353-9d93-e9e00202f79c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e52608a9-aa4f-4353-9d93-e9e00202f79c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:24.577825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:24.578868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:24.579889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:24.579889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:24.579889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:24.579889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:24.579889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:24.579889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:24.579889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:24.579889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:24.579889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:24.579889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:24.579889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:24.579889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:24.579889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 332/646] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:24.579889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52608a9-aa4f-4353-9d93-e9e00202f79c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:24.609312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8bb9b7-f333-40f4-83f0-2679bc74249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:24.609887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bb9b7-f333-40f4-83f0-2679bc74249f None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:24.610164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bb9b7-f333-40f4-83f0-2679bc74249f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:24.610440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bb9b7-f333-40f4-83f0-2679bc74249f None admin] PATH_INFO: `/v3/projects/32bbb936501846e8914acf07890c2891/users/c1f6bb09fd7f4905a1693ef7cb6f088e/roles/5c7e1f557ae04250a01db783f3c27a5f` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:24.625304 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8bb9b7-f333-40f4-83f0-2679bc74249f None admin] RBAC: Authorizing `identity:create_grant(project_id=32bbb936501846e8914acf07890c2891, user_id=c1f6bb09fd7f4905a1693ef7cb6f088e, role_id=5c7e1f557ae04250a01db783f3c27a5f)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:24.626744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8bb9b7-f333-40f4-83f0-2679bc74249f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:24.637861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:24.643668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 315/647] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:15:24 2022] PUT /identity/v3/projects/32bbb936501846e8914acf07890c2891/users/c1f6bb09fd7f4905a1693ef7cb6f088e/roles/5c7e1f557ae04250a01db783f3c27a5f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:15:24.645428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8bb9b7-f333-40f4-83f0-2679bc74249f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:24.693694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52608a9-aa4f-4353-9d93-e9e00202f79c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:24.715593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:24.738330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5b76f-e9eb-4a83-9b46-76aa6a3b8d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:24.739141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5b76f-e9eb-4a83-9b46-76aa6a3b8d22 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:24.739643 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5b76f-e9eb-4a83-9b46-76aa6a3b8d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:24.740018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5b76f-e9eb-4a83-9b46-76aa6a3b8d22 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:24.749990 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5b76f-e9eb-4a83-9b46-76aa6a3b8d22 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:24.752572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5b76f-e9eb-4a83-9b46-76aa6a3b8d22 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b7e5b76f-e9eb-4a83-9b46-76aa6a3b8d22 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:24.767549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:24.768621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:24.769566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:24.769566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:24.769566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:24.769566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:24.769566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:24.769566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:24.769566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:24.769566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:24.769566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 23:15:24.769566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:24.771725 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 333/648] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:15:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:24.772857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5b76f-e9eb-4a83-9b46-76aa6a3b8d22 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:24.778792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8bb9b7-f333-40f4-83f0-2679bc74249f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:24.833428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e50b5-7043-483e-9d2c-20628ce30412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:24.834103 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e50b5-7043-483e-9d2c-20628ce30412 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:24.834454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e50b5-7043-483e-9d2c-20628ce30412 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:24.834799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e50b5-7043-483e-9d2c-20628ce30412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:24.849273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e50b5-7043-483e-9d2c-20628ce30412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:24.849448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e50b5-7043-483e-9d2c-20628ce30412 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-d28e50b5-7043-483e-9d2c-20628ce30412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:24.866357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:24.867411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:24.868396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:24.868396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:24.868396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:24.868396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 316/649] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:24.868396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e50b5-7043-483e-9d2c-20628ce30412 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:24.908200 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5b76f-e9eb-4a83-9b46-76aa6a3b8d22 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:24.926017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:24.950372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b644ff-22de-4558-8f8b-7dbcc0a67923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:24.950992 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b644ff-22de-4558-8f8b-7dbcc0a67923 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:24.951271 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b644ff-22de-4558-8f8b-7dbcc0a67923 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:24.951544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b644ff-22de-4558-8f8b-7dbcc0a67923 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:24.952254 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b644ff-22de-4558-8f8b-7dbcc0a67923 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:24.952560 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b644ff-22de-4558-8f8b-7dbcc0a67923 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:24.954038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b644ff-22de-4558-8f8b-7dbcc0a67923 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:24.967877 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 334/650] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:24.968793 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b644ff-22de-4558-8f8b-7dbcc0a67923 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:25.011274 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:25.016127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e50b5-7043-483e-9d2c-20628ce30412 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:25.060831 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd27ab-eb51-424b-aa10-6e71a7a8d8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:25.061795 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd27ab-eb51-424b-aa10-6e71a7a8d8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:25.062303 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd27ab-eb51-424b-aa10-6e71a7a8d8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:25.062779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd27ab-eb51-424b-aa10-6e71a7a8d8a2 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:25.064114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd27ab-eb51-424b-aa10-6e71a7a8d8a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:25.064607 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd27ab-eb51-424b-aa10-6e71a7a8d8a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:25.066608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd27ab-eb51-424b-aa10-6e71a7a8d8a2 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:25.089977 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 317/651] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Sep 12 23:15:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:25.091109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd27ab-eb51-424b-aa10-6e71a7a8d8a2 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:25.095504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b644ff-22de-4558-8f8b-7dbcc0a67923 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:25.138275 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12a716-b7aa-4f3c-9a5f-f446c412af0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:25.139175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12a716-b7aa-4f3c-9a5f-f446c412af0c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:25.139721 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12a716-b7aa-4f3c-9a5f-f446c412af0c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:25.140507 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12a716-b7aa-4f3c-9a5f-f446c412af0c None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:25.141352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12a716-b7aa-4f3c-9a5f-f446c412af0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:25.142124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12a716-b7aa-4f3c-9a5f-f446c412af0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:25.144011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12a716-b7aa-4f3c-9a5f-f446c412af0c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:25.160828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 335/652] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:25.162107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12a716-b7aa-4f3c-9a5f-f446c412af0c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:25.209653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd27ab-eb51-424b-aa10-6e71a7a8d8a2 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:25.216522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:25.250612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a3af2-787f-4820-8eca-f20e8b4e20f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:25.251287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a3af2-787f-4820-8eca-f20e8b4e20f7 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:25.251670 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a3af2-787f-4820-8eca-f20e8b4e20f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:25.252012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a3af2-787f-4820-8eca-f20e8b4e20f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:25.263242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a3af2-787f-4820-8eca-f20e8b4e20f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:25.264689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a3af2-787f-4820-8eca-f20e8b4e20f7 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-b66a3af2-787f-4820-8eca-f20e8b4e20f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:25.281166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:25.282109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:25.283353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:25.283353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:25.283353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:25.287290 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 318/653] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:25.288213 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a3af2-787f-4820-8eca-f20e8b4e20f7 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:25.302859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:25.305724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12a716-b7aa-4f3c-9a5f-f446c412af0c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:25.346405 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15154d2f-b7a7-481c-aeba-ec9d1db1cb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:25.347052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15154d2f-b7a7-481c-aeba-ec9d1db1cb12 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:25.347413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15154d2f-b7a7-481c-aeba-ec9d1db1cb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:25.347794 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15154d2f-b7a7-481c-aeba-ec9d1db1cb12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:25.412603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a3af2-787f-4820-8eca-f20e8b4e20f7 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:25.440447 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15154d2f-b7a7-481c-aeba-ec9d1db1cb12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:25.442853 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15154d2f-b7a7-481c-aeba-ec9d1db1cb12 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:25.450971 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9b72b-b65a-4d33-97e1-1640d7e1d16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:25.451707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9b72b-b65a-4d33-97e1-1640d7e1d16d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:25.452052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9b72b-b65a-4d33-97e1-1640d7e1d16d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:25.452396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9b72b-b65a-4d33-97e1-1640d7e1d16d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:25.460223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9b72b-b65a-4d33-97e1-1640d7e1d16d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:25.461809 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9b72b-b65a-4d33-97e1-1640d7e1d16d None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-94e9b72b-b65a-4d33-97e1-1640d7e1d16d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:25.473368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:25.474326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:25.475271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:25.475271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:25.475271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:25.475271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:25.475271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:25.475271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:25.475271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:25.475271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:25.475271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:15:25.475271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:25.479288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 319/654] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:15:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:25.480357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9b72b-b65a-4d33-97e1-1640d7e1d16d None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:25.489018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:25.574799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:25.575260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 336/655] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:15:25.576431 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15154d2f-b7a7-481c-aeba-ec9d1db1cb12 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:25.618931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9b72b-b65a-4d33-97e1-1640d7e1d16d None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:25.662956 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac847c-7de1-44db-8dee-491a7f64c6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:25.663617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac847c-7de1-44db-8dee-491a7f64c6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:25.664002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac847c-7de1-44db-8dee-491a7f64c6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:25.664375 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac847c-7de1-44db-8dee-491a7f64c6d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:25.665483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac847c-7de1-44db-8dee-491a7f64c6d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:25.665861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac847c-7de1-44db-8dee-491a7f64c6d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:25.667596 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac847c-7de1-44db-8dee-491a7f64c6d9 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:25.689806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 320/656] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:25.690650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac847c-7de1-44db-8dee-491a7f64c6d9 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:25.691845 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15154d2f-b7a7-481c-aeba-ec9d1db1cb12 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:25.732665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a8142-c3f2-4826-9b7d-fed02cd54243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:25.733681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a8142-c3f2-4826-9b7d-fed02cd54243 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:25.734535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a8142-c3f2-4826-9b7d-fed02cd54243 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:25.735035 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a8142-c3f2-4826-9b7d-fed02cd54243 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:25.736295 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a8142-c3f2-4826-9b7d-fed02cd54243 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:25.737028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a8142-c3f2-4826-9b7d-fed02cd54243 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:25.738612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a8142-c3f2-4826-9b7d-fed02cd54243 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:25.740458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:25.760188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 337/657] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Sep 12 23:15:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:25.761250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a8142-c3f2-4826-9b7d-fed02cd54243 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:25.802813 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac847c-7de1-44db-8dee-491a7f64c6d9 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:25.834136 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:25.842448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d651b31-d6af-4b44-9773-1d2a58e78b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:25.843149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d651b31-d6af-4b44-9773-1d2a58e78b9f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:25.843492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d651b31-d6af-4b44-9773-1d2a58e78b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:25.843869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d651b31-d6af-4b44-9773-1d2a58e78b9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:25.855304 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d651b31-d6af-4b44-9773-1d2a58e78b9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:25.856919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d651b31-d6af-4b44-9773-1d2a58e78b9f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:25.872365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a8142-c3f2-4826-9b7d-fed02cd54243 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-1d651b31-d6af-4b44-9773-1d2a58e78b9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:25.873002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:25.873927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:25.874819 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:25.874819 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:25.874819 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:25.878861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 321/658] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:25.879847 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d651b31-d6af-4b44-9773-1d2a58e78b9f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:25.920034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a6e23-45f1-497d-9548-dae06e2e1786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:25.920684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a6e23-45f1-497d-9548-dae06e2e1786 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:25.921637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a6e23-45f1-497d-9548-dae06e2e1786 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:25.921869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a6e23-45f1-497d-9548-dae06e2e1786 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:25.931494 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a6e23-45f1-497d-9548-dae06e2e1786 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:25.933363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a6e23-45f1-497d-9548-dae06e2e1786 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b14a6e23-45f1-497d-9548-dae06e2e1786 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:25.947356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:25.948347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:25.949280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:25.949280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:25.949280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:25.949280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:25.949280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:25.949280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:25.949280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:25.949280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:25.949280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 23:15:25.949280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:25.952117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 338/659] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:25.953230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a6e23-45f1-497d-9548-dae06e2e1786 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:25.996408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d651b31-d6af-4b44-9773-1d2a58e78b9f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:26.022415 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:26.036423 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48da8a-3fcb-4697-bd20-ba913f9b0818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:26.037205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48da8a-3fcb-4697-bd20-ba913f9b0818 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:26.037581 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48da8a-3fcb-4697-bd20-ba913f9b0818 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:26.037949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48da8a-3fcb-4697-bd20-ba913f9b0818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:26.084920 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a6e23-45f1-497d-9548-dae06e2e1786 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:26.090132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:26.112759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48da8a-3fcb-4697-bd20-ba913f9b0818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:26.114399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48da8a-3fcb-4697-bd20-ba913f9b0818 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:26.128533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771add8-1c42-42c9-8e0c-94b9f4e1b80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:26.129703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771add8-1c42-42c9-8e0c-94b9f4e1b80e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:26.130607 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771add8-1c42-42c9-8e0c-94b9f4e1b80e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:26.131035 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771add8-1c42-42c9-8e0c-94b9f4e1b80e None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:26.132121 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771add8-1c42-42c9-8e0c-94b9f4e1b80e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:26.132577 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771add8-1c42-42c9-8e0c-94b9f4e1b80e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:26.134830 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771add8-1c42-42c9-8e0c-94b9f4e1b80e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:26.150462 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 339/660] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:26.151474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771add8-1c42-42c9-8e0c-94b9f4e1b80e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:26.228013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 322/661] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:15:26.228986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48da8a-3fcb-4697-bd20-ba913f9b0818 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:26.272581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771add8-1c42-42c9-8e0c-94b9f4e1b80e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:26.288146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:26.319361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfe44c-3d66-4a15-b046-8701ed6e9540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:26.321679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfe44c-3d66-4a15-b046-8701ed6e9540 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:26.322121 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfe44c-3d66-4a15-b046-8701ed6e9540 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:26.322544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfe44c-3d66-4a15-b046-8701ed6e9540 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:26.323458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfe44c-3d66-4a15-b046-8701ed6e9540 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:26.323946 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfe44c-3d66-4a15-b046-8701ed6e9540 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:26.325536 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfe44c-3d66-4a15-b046-8701ed6e9540 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:26.334118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:26.352958 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 340/662] 10.222.0.27 () {56 vars in 1139 bytes} [Mon Sep 12 23:15:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:26.354447 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfe44c-3d66-4a15-b046-8701ed6e9540 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:26.355485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48da8a-3fcb-4697-bd20-ba913f9b0818 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:26.393364 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88070bc4-4f60-49e4-a0d6-b4f16fc72d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:26.394209 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88070bc4-4f60-49e4-a0d6-b4f16fc72d70 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:26.394605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88070bc4-4f60-49e4-a0d6-b4f16fc72d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:26.395250 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88070bc4-4f60-49e4-a0d6-b4f16fc72d70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:26.403144 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88070bc4-4f60-49e4-a0d6-b4f16fc72d70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:26.405501 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88070bc4-4f60-49e4-a0d6-b4f16fc72d70 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-88070bc4-4f60-49e4-a0d6-b4f16fc72d70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:26.418534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:26.419711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:26.420800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:26.420800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:26.420800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:26.420800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:26.420800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:26.420800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:26.420800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:26.420800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:26.426923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 323/663] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:26.427960 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88070bc4-4f60-49e4-a0d6-b4f16fc72d70 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:26.485523 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfe44c-3d66-4a15-b046-8701ed6e9540 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:26.526628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279ad70-30bb-4970-84e7-4c8930de4bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:26.527263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279ad70-30bb-4970-84e7-4c8930de4bda None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:26.527932 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279ad70-30bb-4970-84e7-4c8930de4bda None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:26.528223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279ad70-30bb-4970-84e7-4c8930de4bda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:26.539809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279ad70-30bb-4970-84e7-4c8930de4bda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:26.541479 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279ad70-30bb-4970-84e7-4c8930de4bda None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8279ad70-30bb-4970-84e7-4c8930de4bda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:26.556049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:26.557178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:26.558061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:26.558061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:26.558061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:26.558061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 341/664] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:26.558061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279ad70-30bb-4970-84e7-4c8930de4bda None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:26.564381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88070bc4-4f60-49e4-a0d6-b4f16fc72d70 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:26.585526 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:26.599424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530844f-ce1c-4273-9dc7-d2d48ba2c873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:26.600082 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530844f-ce1c-4273-9dc7-d2d48ba2c873 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:26.600364 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530844f-ce1c-4273-9dc7-d2d48ba2c873 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:26.600635 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530844f-ce1c-4273-9dc7-d2d48ba2c873 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:26.612780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:26.668667 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279ad70-30bb-4970-84e7-4c8930de4bda None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:26.672462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530844f-ce1c-4273-9dc7-d2d48ba2c873 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:26.673922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530844f-ce1c-4273-9dc7-d2d48ba2c873 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:26.705173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3a845-bb81-4442-b648-58a59990e6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:26.706039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3a845-bb81-4442-b648-58a59990e6bd None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:26.706039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3a845-bb81-4442-b648-58a59990e6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:26.706331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3a845-bb81-4442-b648-58a59990e6bd None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:26.707035 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3a845-bb81-4442-b648-58a59990e6bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:26.707345 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3a845-bb81-4442-b648-58a59990e6bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:26.708792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3a845-bb81-4442-b648-58a59990e6bd None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:26.726636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 342/665] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:26.726636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37572554-81c5-463b-96dd-25c8948d5544 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:26.726636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37572554-81c5-463b-96dd-25c8948d5544 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:26.726636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37572554-81c5-463b-96dd-25c8948d5544 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:26.726636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 343/666] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:26.727065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37572554-81c5-463b-96dd-25c8948d5544 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:26.787851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 324/667] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:15:26.792413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530844f-ce1c-4273-9dc7-d2d48ba2c873 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:26.822303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:26.840573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:26.842722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37572554-81c5-463b-96dd-25c8948d5544 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:26.878648 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7057a-0a8b-4cbb-8e0e-1f8752aede71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:26.879289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7057a-0a8b-4cbb-8e0e-1f8752aede71 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:26.879613 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7057a-0a8b-4cbb-8e0e-1f8752aede71 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:26.879933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7057a-0a8b-4cbb-8e0e-1f8752aede71 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:26.880602 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7057a-0a8b-4cbb-8e0e-1f8752aede71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:26.880913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7057a-0a8b-4cbb-8e0e-1f8752aede71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:26.882277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7057a-0a8b-4cbb-8e0e-1f8752aede71 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:26.893960 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 344/668] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:26.895100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e423d1a-3725-4b81-8a45-ca93b29acd6e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:26.895385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e423d1a-3725-4b81-8a45-ca93b29acd6e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:26.895689 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e423d1a-3725-4b81-8a45-ca93b29acd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:26.902715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530844f-ce1c-4273-9dc7-d2d48ba2c873 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:26.914036 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6e423d1a-3725-4b81-8a45-ca93b29acd6e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:26.917069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6e423d1a-3725-4b81-8a45-ca93b29acd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:26.939361 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046aa48-771d-4f3a-b537-58bcfd559fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:26.940019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046aa48-771d-4f3a-b537-58bcfd559fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:26.940296 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046aa48-771d-4f3a-b537-58bcfd559fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:26.940566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046aa48-771d-4f3a-b537-58bcfd559fa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:26.941278 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046aa48-771d-4f3a-b537-58bcfd559fa7 None admin] RBAC: Adding query filter params (role.id=5c7e1f557ae04250a01db783f3c27a5f, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=d61407406ef242a78030f79532fe2247) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:26.941578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046aa48-771d-4f3a-b537-58bcfd559fa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:26.942991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046aa48-771d-4f3a-b537-58bcfd559fa7 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:26.949101 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 325/669] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:26 2022] GET /identity/v3/role_assignments?user.id=d61407406ef242a78030f79532fe2247&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=5c7e1f557ae04250a01db783f3c27a5f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:26.949985 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046aa48-771d-4f3a-b537-58bcfd559fa7 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:26.999727 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 345/670] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:26.999727 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c8ebd-8879-4c1f-92aa-067a126366b2 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:26.999727 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c8ebd-8879-4c1f-92aa-067a126366b2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:26.999727 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c8ebd-8879-4c1f-92aa-067a126366b2 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:26.999727 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 346/671] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:26.999727 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c8ebd-8879-4c1f-92aa-067a126366b2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:27.096867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046aa48-771d-4f3a-b537-58bcfd559fa7 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:27.105774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:27.108229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:27.144785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c8ebd-8879-4c1f-92aa-067a126366b2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:27.145523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbdcf4-f9cd-4c0d-a5de-1705f0c488ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:27.146317 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbdcf4-f9cd-4c0d-a5de-1705f0c488ad None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:27.146608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbdcf4-f9cd-4c0d-a5de-1705f0c488ad None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:27.146880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbdcf4-f9cd-4c0d-a5de-1705f0c488ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:27.162683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbdcf4-f9cd-4c0d-a5de-1705f0c488ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:27.164245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbdcf4-f9cd-4c0d-a5de-1705f0c488ad None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-16dbdcf4-f9cd-4c0d-a5de-1705f0c488ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:27.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:27.180204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:27.181099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:27.181099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:27.181099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:27.181750 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 326/672] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:27.182910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b5792-ec82-4ed6-be17-65aa1bc09335 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:27.182910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b5792-ec82-4ed6-be17-65aa1bc09335 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:27.183233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b5792-ec82-4ed6-be17-65aa1bc09335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:27.190760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16dd808-e872-4e5b-9ba4-cf2f8e9dc326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:27.191485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dd808-e872-4e5b-9ba4-cf2f8e9dc326 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:27.191825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dd808-e872-4e5b-9ba4-cf2f8e9dc326 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:27.192101 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dd808-e872-4e5b-9ba4-cf2f8e9dc326 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:27.200008 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16dd808-e872-4e5b-9ba4-cf2f8e9dc326 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:27.201463 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16dd808-e872-4e5b-9ba4-cf2f8e9dc326 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:27.201876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-aa2b5792-ec82-4ed6-be17-65aa1bc09335 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:27.203063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-aa2b5792-ec82-4ed6-be17-65aa1bc09335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f16dd808-e872-4e5b-9ba4-cf2f8e9dc326 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:27.213709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:27.214609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:27.215644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:27.215644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:27.215644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:27.215644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:27.215644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:27.215644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:27.215644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:27.215644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:27.215644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 347/673] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:27.215644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16dd808-e872-4e5b-9ba4-cf2f8e9dc326 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:27.292693 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 327/674] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:27.293891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b5792-ec82-4ed6-be17-65aa1bc09335 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:27.345500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16dd808-e872-4e5b-9ba4-cf2f8e9dc326 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:27.382312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:27.394379 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e580b9-6f6a-47c9-8a87-506699199d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:27.395077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e580b9-6f6a-47c9-8a87-506699199d8c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:27.395455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e580b9-6f6a-47c9-8a87-506699199d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:27.395840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e580b9-6f6a-47c9-8a87-506699199d8c None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:27.396650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:27.397572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e580b9-6f6a-47c9-8a87-506699199d8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:27.398205 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e580b9-6f6a-47c9-8a87-506699199d8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:27.400485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e580b9-6f6a-47c9-8a87-506699199d8c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:27.410515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b5792-ec82-4ed6-be17-65aa1bc09335 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:27.415998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 348/675] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:27.416841 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e580b9-6f6a-47c9-8a87-506699199d8c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:27.454829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3d6da-43b9-4af3-811a-da7677b39dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:27.455474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3d6da-43b9-4af3-811a-da7677b39dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:27.455793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3d6da-43b9-4af3-811a-da7677b39dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:27.456355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3d6da-43b9-4af3-811a-da7677b39dc1 None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:27.457988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3d6da-43b9-4af3-811a-da7677b39dc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:27.458355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3d6da-43b9-4af3-811a-da7677b39dc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:27.459888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3d6da-43b9-4af3-811a-da7677b39dc1 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:27.476573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 328/676] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:27.477428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3d6da-43b9-4af3-811a-da7677b39dc1 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:27.535889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e580b9-6f6a-47c9-8a87-506699199d8c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:27.572308 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97834ad1-cf0f-40e7-89f0-0b4c2ee684c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:27.572938 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97834ad1-cf0f-40e7-89f0-0b4c2ee684c4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:27.573272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97834ad1-cf0f-40e7-89f0-0b4c2ee684c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:27.573546 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97834ad1-cf0f-40e7-89f0-0b4c2ee684c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:27.584440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97834ad1-cf0f-40e7-89f0-0b4c2ee684c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-97834ad1-cf0f-40e7-89f0-0b4c2ee684c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:27.589924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:27.592102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:27.593967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:27.593967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:27.593967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:27.593967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:27.593967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:27.593967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:27.593967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:27.593967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:27.593967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:27.593967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:27.593967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:15:27.593967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:27.593967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 349/677] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:15:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:27.593967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97834ad1-cf0f-40e7-89f0-0b4c2ee684c4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:27.615898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3d6da-43b9-4af3-811a-da7677b39dc1 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:27.647587 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:27.650744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7c979-c7e2-4153-a658-56e3b25aa7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:27.651389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7c979-c7e2-4153-a658-56e3b25aa7da None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:27.651697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7c979-c7e2-4153-a658-56e3b25aa7da None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:27.651977 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7c979-c7e2-4153-a658-56e3b25aa7da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:27.659092 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7c979-c7e2-4153-a658-56e3b25aa7da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:27.660669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7c979-c7e2-4153-a658-56e3b25aa7da None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-eca7c979-c7e2-4153-a658-56e3b25aa7da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:27.672658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:27.673612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:27.674520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:27.674520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:27.674520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:27.674520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:27.674520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:27.674520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:27.674520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:27.674520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:27.675934 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 329/678] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:27.677050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7c979-c7e2-4153-a658-56e3b25aa7da None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:27.696034 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:27.716236 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97834ad1-cf0f-40e7-89f0-0b4c2ee684c4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:27.752488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ba6aa-233a-4db5-97e3-6492451dcf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:27.753097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ba6aa-233a-4db5-97e3-6492451dcf4b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:27.753374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ba6aa-233a-4db5-97e3-6492451dcf4b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:27.753642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ba6aa-233a-4db5-97e3-6492451dcf4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:27.754360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ba6aa-233a-4db5-97e3-6492451dcf4b None admin] RBAC: Adding query filter params (role.id=5c7e1f557ae04250a01db783f3c27a5f, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=c5b8828077594349a93c7a842c0f15da) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:27.754666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ba6aa-233a-4db5-97e3-6492451dcf4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:27.756104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ba6aa-233a-4db5-97e3-6492451dcf4b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:27.771601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 350/679] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:27 2022] GET /identity/v3/role_assignments?user.id=c5b8828077594349a93c7a842c0f15da&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=5c7e1f557ae04250a01db783f3c27a5f => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:27.771601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ba6aa-233a-4db5-97e3-6492451dcf4b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:27.827000 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7c979-c7e2-4153-a658-56e3b25aa7da None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:27.883929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543b8e4-2bad-4d71-9985-584706184c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:27.884595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543b8e4-2bad-4d71-9985-584706184c27 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:27.884973 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543b8e4-2bad-4d71-9985-584706184c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:27.885382 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543b8e4-2bad-4d71-9985-584706184c27 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:27.886186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543b8e4-2bad-4d71-9985-584706184c27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:27.886560 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543b8e4-2bad-4d71-9985-584706184c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:27.888194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543b8e4-2bad-4d71-9985-584706184c27 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:27.893898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ba6aa-233a-4db5-97e3-6492451dcf4b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:27.900503 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 330/680] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:15:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:27.901488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543b8e4-2bad-4d71-9985-584706184c27 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:27.915656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:27.935114 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b075d38-64c9-4425-917c-0bde1d4f4f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:27.935950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b075d38-64c9-4425-917c-0bde1d4f4f8b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:27.936358 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b075d38-64c9-4425-917c-0bde1d4f4f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:27.936725 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b075d38-64c9-4425-917c-0bde1d4f4f8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:27.948875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b075d38-64c9-4425-917c-0bde1d4f4f8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-9b075d38-64c9-4425-917c-0bde1d4f4f8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:27.955443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:27.956690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:27.957732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:27.957732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:27.957732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:27.957732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:27.957732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:27.957732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:27.957732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:27.957732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:27.957732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:27.957732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:27.957732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:27.957732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:27.964413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 351/681] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:27.964413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b075d38-64c9-4425-917c-0bde1d4f4f8b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:28.015846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:28.062380 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543b8e4-2bad-4d71-9985-584706184c27 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:28.093097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b075d38-64c9-4425-917c-0bde1d4f4f8b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:28.121474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afb8bb-d36c-4fd0-be8c-e6ece4fc5d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:28.122477 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afb8bb-d36c-4fd0-be8c-e6ece4fc5d2e None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:28.122993 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afb8bb-d36c-4fd0-be8c-e6ece4fc5d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:28.123462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afb8bb-d36c-4fd0-be8c-e6ece4fc5d2e None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:28.124382 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afb8bb-d36c-4fd0-be8c-e6ece4fc5d2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:28.124888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afb8bb-d36c-4fd0-be8c-e6ece4fc5d2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:28.126636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afb8bb-d36c-4fd0-be8c-e6ece4fc5d2e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:28.131356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845f98b-0362-4f8a-a81d-d55c381c19cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:28.132044 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f98b-0362-4f8a-a81d-d55c381c19cd None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:28.132330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f98b-0362-4f8a-a81d-d55c381c19cd None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:28.132640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f98b-0362-4f8a-a81d-d55c381c19cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:28.141346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 331/682] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:28.142470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afb8bb-d36c-4fd0-be8c-e6ece4fc5d2e None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:28.145347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845f98b-0362-4f8a-a81d-d55c381c19cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:28.147152 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845f98b-0362-4f8a-a81d-d55c381c19cd None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-9845f98b-0362-4f8a-a81d-d55c381c19cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:28.165333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:28.166333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:28.167230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:28.167230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:28.167230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:28.167230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 352/683] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:28.167230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845f98b-0362-4f8a-a81d-d55c381c19cd None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:28.204032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:28.281157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afb8bb-d36c-4fd0-be8c-e6ece4fc5d2e None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:28.283882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845f98b-0362-4f8a-a81d-d55c381c19cd None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:28.321145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:28.321342 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe7327-b073-4e51-8906-5816b71d01e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:28.322178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe7327-b073-4e51-8906-5816b71d01e4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:28.322178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe7327-b073-4e51-8906-5816b71d01e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:28.322486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe7327-b073-4e51-8906-5816b71d01e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:28.323312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe7327-b073-4e51-8906-5816b71d01e4 None admin] RBAC: Adding query filter params (role.id=5c7e1f557ae04250a01db783f3c27a5f, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=bc6e63e9fc3447d5a894a777a98fe773) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:28.323654 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe7327-b073-4e51-8906-5816b71d01e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:28.326100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe7327-b073-4e51-8906-5816b71d01e4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:28.330171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9412cb9-3b63-458f-8aae-577de5a61699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:28.330783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9412cb9-3b63-458f-8aae-577de5a61699 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:28.331063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9412cb9-3b63-458f-8aae-577de5a61699 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:28.331340 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9412cb9-3b63-458f-8aae-577de5a61699 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:28.332041 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9412cb9-3b63-458f-8aae-577de5a61699 None admin] RBAC: Adding query filter params (name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:28.332344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9412cb9-3b63-458f-8aae-577de5a61699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:28.333787 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9412cb9-3b63-458f-8aae-577de5a61699 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:28.339896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 353/684] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:28 2022] GET /identity/v3/role_assignments?user.id=bc6e63e9fc3447d5a894a777a98fe773&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=5c7e1f557ae04250a01db783f3c27a5f => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:28.340930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe7327-b073-4e51-8906-5816b71d01e4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:28.343717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 332/685] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:28.349418 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9412cb9-3b63-458f-8aae-577de5a61699 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:28.478981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:28.491751 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe7327-b073-4e51-8906-5816b71d01e4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:28.534477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6298bd-b889-4a61-b9b3-6f1c50c97563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:28.535121 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6298bd-b889-4a61-b9b3-6f1c50c97563 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:28.535869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6298bd-b889-4a61-b9b3-6f1c50c97563 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:28.536152 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6298bd-b889-4a61-b9b3-6f1c50c97563 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:28.537213 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6298bd-b889-4a61-b9b3-6f1c50c97563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:28.537595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6298bd-b889-4a61-b9b3-6f1c50c97563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:28.539861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6298bd-b889-4a61-b9b3-6f1c50c97563 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:28.555147 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 354/686] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:28.560739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6298bd-b889-4a61-b9b3-6f1c50c97563 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:28.562002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9412cb9-3b63-458f-8aae-577de5a61699 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:28.603982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413fea4-156f-4bcd-8179-d8410b1190ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:28.604621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413fea4-156f-4bcd-8179-d8410b1190ef None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:28.604901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413fea4-156f-4bcd-8179-d8410b1190ef None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:28.605169 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413fea4-156f-4bcd-8179-d8410b1190ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:28.616221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413fea4-156f-4bcd-8179-d8410b1190ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:28.617699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413fea4-156f-4bcd-8179-d8410b1190ef None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-4413fea4-156f-4bcd-8179-d8410b1190ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:28.635106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:28.636151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:28.637051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:28.637051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:28.637051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:28.637729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 333/687] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:28.642911 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413fea4-156f-4bcd-8179-d8410b1190ef None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:28.683580 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6298bd-b889-4a61-b9b3-6f1c50c97563 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:28.694794 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:28.728168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888601a-1f0b-43b7-91a6-a1e433a22502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:28.728867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888601a-1f0b-43b7-91a6-a1e433a22502 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:28.729209 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888601a-1f0b-43b7-91a6-a1e433a22502 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:28.729427 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888601a-1f0b-43b7-91a6-a1e433a22502 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:28.738352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888601a-1f0b-43b7-91a6-a1e433a22502 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:28.740194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888601a-1f0b-43b7-91a6-a1e433a22502 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a888601a-1f0b-43b7-91a6-a1e433a22502 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:28.757360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:28.759762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:28.761767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:28.761767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:28.761767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:28.761767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:28.761767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:28.761767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:28.761767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:28.761767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:28.761767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:15:28.761767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:28.761767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 355/688] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Sep 12 23:15:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:28.762511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413fea4-156f-4bcd-8179-d8410b1190ef None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:28.769874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888601a-1f0b-43b7-91a6-a1e433a22502 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:28.791943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:28.798095 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b91c13-edfc-4737-bcd9-87e91be68ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:28.798707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b91c13-edfc-4737-bcd9-87e91be68ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:28.798985 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b91c13-edfc-4737-bcd9-87e91be68ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:28.799253 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b91c13-edfc-4737-bcd9-87e91be68ad5 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:28.800041 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b91c13-edfc-4737-bcd9-87e91be68ad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:28.800400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b91c13-edfc-4737-bcd9-87e91be68ad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:28.801772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b91c13-edfc-4737-bcd9-87e91be68ad5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:28.821888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 334/689] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:28.827247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b91c13-edfc-4737-bcd9-87e91be68ad5 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:28.893459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888601a-1f0b-43b7-91a6-a1e433a22502 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:28.934841 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d0de3-a521-490d-95e5-5ac389724b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:28.935459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0de3-a521-490d-95e5-5ac389724b4a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:28.935773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0de3-a521-490d-95e5-5ac389724b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:28.936048 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0de3-a521-490d-95e5-5ac389724b4a None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:28.937068 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d0de3-a521-490d-95e5-5ac389724b4a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:28.937376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d0de3-a521-490d-95e5-5ac389724b4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:28.938847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d0de3-a521-490d-95e5-5ac389724b4a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:28.962614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 356/690] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Sep 12 23:15:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:28.968211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d0de3-a521-490d-95e5-5ac389724b4a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:29.063315 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b91c13-edfc-4737-bcd9-87e91be68ad5 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:29.067833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:29.109518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:29.144112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d0de3-a521-490d-95e5-5ac389724b4a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:29.148293 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592ba4-2d1b-4a92-b4c5-397677fcd74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:29.148939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592ba4-2d1b-4a92-b4c5-397677fcd74a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:29.149219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592ba4-2d1b-4a92-b4c5-397677fcd74a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:29.149494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592ba4-2d1b-4a92-b4c5-397677fcd74a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:29.165163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3592ba4-2d1b-4a92-b4c5-397677fcd74a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:29.166608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3592ba4-2d1b-4a92-b4c5-397677fcd74a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:29.181805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7baf0-53df-4ed7-88ea-a7fb1e439e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:29.182423 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7baf0-53df-4ed7-88ea-a7fb1e439e9c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:29.182705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7baf0-53df-4ed7-88ea-a7fb1e439e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:29.182981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7baf0-53df-4ed7-88ea-a7fb1e439e9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-d3592ba4-2d1b-4a92-b4c5-397677fcd74a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:29.189191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:29.191121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:29.192417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:29.192417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:29.192417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:29.192417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:29.192417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:29.192417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:29.192417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:29.192417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:29.192417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:15:29.192417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:29.192417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 335/691] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:15:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:15:29.200761 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592ba4-2d1b-4a92-b4c5-397677fcd74a None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:29.306078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7baf0-53df-4ed7-88ea-a7fb1e439e9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:29.307530 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7baf0-53df-4ed7-88ea-a7fb1e439e9c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:29.395096 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592ba4-2d1b-4a92-b4c5-397677fcd74a None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:29.405394 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:29.439491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efdcad-f7e5-4288-b479-e5fcd7562e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:29.440365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efdcad-f7e5-4288-b479-e5fcd7562e47 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:29.440789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efdcad-f7e5-4288-b479-e5fcd7562e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:29.441242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 357/692] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:15:29.441540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efdcad-f7e5-4288-b479-e5fcd7562e47 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:29.442446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49efdcad-f7e5-4288-b479-e5fcd7562e47 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:29.442966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49efdcad-f7e5-4288-b479-e5fcd7562e47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:29.444606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49efdcad-f7e5-4288-b479-e5fcd7562e47 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:29.447143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af05231-3fc4-4894-82ae-959d0df745c6 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:29.447497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af05231-3fc4-4894-82ae-959d0df745c6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:29.447881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af05231-3fc4-4894-82ae-959d0df745c6 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:29.453468 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 358/693] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:29.453468 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af05231-3fc4-4894-82ae-959d0df745c6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:29.467611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 336/694] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Sep 12 23:15:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:29.468805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e828b11-d076-4ef7-8124-36ed4aceab62 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:29.469085 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e828b11-d076-4ef7-8124-36ed4aceab62 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:29.469444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e828b11-d076-4ef7-8124-36ed4aceab62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:29.488957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-5e828b11-d076-4ef7-8124-36ed4aceab62 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:29.490276 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5e828b11-d076-4ef7-8124-36ed4aceab62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:29.508105 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:29.575601 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 337/695] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:29.575601 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e828b11-d076-4ef7-8124-36ed4aceab62 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:29.576001 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af05231-3fc4-4894-82ae-959d0df745c6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:29.625421 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc6e62-f881-4d0b-aa6c-b0cc851fbf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:29.626054 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc6e62-f881-4d0b-aa6c-b0cc851fbf7c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:29.626331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc6e62-f881-4d0b-aa6c-b0cc851fbf7c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:29.626600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc6e62-f881-4d0b-aa6c-b0cc851fbf7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:29.643193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc6e62-f881-4d0b-aa6c-b0cc851fbf7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:29.645896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc6e62-f881-4d0b-aa6c-b0cc851fbf7c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-85bc6e62-f881-4d0b-aa6c-b0cc851fbf7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:29.669019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:29.670017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:29.670911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:29.670911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:29.670911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:29.670911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 359/696] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:29.677243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc6e62-f881-4d0b-aa6c-b0cc851fbf7c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:29.733651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:29.781714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e828b11-d076-4ef7-8124-36ed4aceab62 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:29.854592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89d745-452d-4893-acb0-5ce580caa7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:29.855215 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89d745-452d-4893-acb0-5ce580caa7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:29.855502 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89d745-452d-4893-acb0-5ce580caa7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:29.855805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89d745-452d-4893-acb0-5ce580caa7b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:29.870972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc6e62-f881-4d0b-aa6c-b0cc851fbf7c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:29.910171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:29.921783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e610ba-5618-442a-8dd1-45bd010ccddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:29.922404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e610ba-5618-442a-8dd1-45bd010ccddb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:29.922687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e610ba-5618-442a-8dd1-45bd010ccddb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:29.922962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e610ba-5618-442a-8dd1-45bd010ccddb None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:29.923628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e610ba-5618-442a-8dd1-45bd010ccddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:29.924027 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e610ba-5618-442a-8dd1-45bd010ccddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:29.925944 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e610ba-5618-442a-8dd1-45bd010ccddb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:29.945724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 360/697] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:29.946533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e610ba-5618-442a-8dd1-45bd010ccddb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:29.980737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89d745-452d-4893-acb0-5ce580caa7b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:29.983003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89d745-452d-4893-acb0-5ce580caa7b5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:30.041935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:30.061904 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e610ba-5618-442a-8dd1-45bd010ccddb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:30.103289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70089419-6b03-4ca9-a4cf-25bace624327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:30.103965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70089419-6b03-4ca9-a4cf-25bace624327 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:30.104253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70089419-6b03-4ca9-a4cf-25bace624327 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:30.104534 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70089419-6b03-4ca9-a4cf-25bace624327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:30.115501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70089419-6b03-4ca9-a4cf-25bace624327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:30.117085 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70089419-6b03-4ca9-a4cf-25bace624327 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:30.122279 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 338/698] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:15:30.123762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89d745-452d-4893-acb0-5ce580caa7b5 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-70089419-6b03-4ca9-a4cf-25bace624327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:30.133972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:30.134936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:30.135915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:30.135915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:30.135915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:30.140132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 361/699] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:30.140891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70089419-6b03-4ca9-a4cf-25bace624327 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:30.197800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:30.256790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70089419-6b03-4ca9-a4cf-25bace624327 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:30.269546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89d745-452d-4893-acb0-5ce580caa7b5 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:30.304071 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd018290-8ae5-4e4e-b97b-5a6d11ee5a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:30.306249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd018290-8ae5-4e4e-b97b-5a6d11ee5a6f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:30.307022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd018290-8ae5-4e4e-b97b-5a6d11ee5a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:30.307503 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd018290-8ae5-4e4e-b97b-5a6d11ee5a6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:30.309663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:30.311216 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7baf8e-51f1-4c67-91cf-683f834ddefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:30.312042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7baf8e-51f1-4c67-91cf-683f834ddefc None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:30.312332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7baf8e-51f1-4c67-91cf-683f834ddefc None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:30.312613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7baf8e-51f1-4c67-91cf-683f834ddefc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:30.320142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7baf8e-51f1-4c67-91cf-683f834ddefc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:30.321681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7baf8e-51f1-4c67-91cf-683f834ddefc None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:30.322363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd018290-8ae5-4e4e-b97b-5a6d11ee5a6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:30.324730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd018290-8ae5-4e4e-b97b-5a6d11ee5a6f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-5c7baf8e-51f1-4c67-91cf-683f834ddefc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:30.333919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:30.334874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:30.336352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:30.336352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:30.336352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:30.336352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:30.336352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:30.336352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:30.336352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:30.336352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:30.336352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 339/700] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:30.336867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7baf8e-51f1-4c67-91cf-683f834ddefc None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-fd018290-8ae5-4e4e-b97b-5a6d11ee5a6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:30.344967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:30.346010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:30.346897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:30.346897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:30.346897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:30.348928 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 362/701] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:30.354279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd018290-8ae5-4e4e-b97b-5a6d11ee5a6f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:30.456862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7baf8e-51f1-4c67-91cf-683f834ddefc None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:30.468206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd018290-8ae5-4e4e-b97b-5a6d11ee5a6f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:30.472165 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:30.496608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366da40d-85bb-4846-adbe-99348803a02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:30.497213 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366da40d-85bb-4846-adbe-99348803a02d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:30.497493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366da40d-85bb-4846-adbe-99348803a02d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:30.497769 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366da40d-85bb-4846-adbe-99348803a02d None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:30.498408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366da40d-85bb-4846-adbe-99348803a02d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:30.498715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366da40d-85bb-4846-adbe-99348803a02d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:30.500185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366da40d-85bb-4846-adbe-99348803a02d None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:30.507873 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86819299-2caa-487e-a158-ad5c65a0bd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:30.508481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86819299-2caa-487e-a158-ad5c65a0bd75 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:30.508766 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86819299-2caa-487e-a158-ad5c65a0bd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:30.509047 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86819299-2caa-487e-a158-ad5c65a0bd75 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:30.509741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86819299-2caa-487e-a158-ad5c65a0bd75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:30.510053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86819299-2caa-487e-a158-ad5c65a0bd75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:30.511474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86819299-2caa-487e-a158-ad5c65a0bd75 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:30.512374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 340/702] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:30.513079 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366da40d-85bb-4846-adbe-99348803a02d None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:30.531786 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 363/703] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:30.533466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ab135-47f4-4d17-a45c-99e7ace4fc39 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:30.533826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ab135-47f4-4d17-a45c-99e7ace4fc39 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:30.534179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ab135-47f4-4d17-a45c-99e7ace4fc39 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:30.535760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 364/704] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:30.536534 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ab135-47f4-4d17-a45c-99e7ace4fc39 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:30.593569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:30.628846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366da40d-85bb-4846-adbe-99348803a02d None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:30.653267 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ab135-47f4-4d17-a45c-99e7ace4fc39 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:30.665597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ef59d-bfb9-4bc1-b00c-be8831abeb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:30.666648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ef59d-bfb9-4bc1-b00c-be8831abeb4b None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:30.667199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ef59d-bfb9-4bc1-b00c-be8831abeb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:30.667490 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ef59d-bfb9-4bc1-b00c-be8831abeb4b None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:30.668197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ef59d-bfb9-4bc1-b00c-be8831abeb4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:30.668763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ef59d-bfb9-4bc1-b00c-be8831abeb4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:30.670868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ef59d-bfb9-4bc1-b00c-be8831abeb4b None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:30.689796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 341/705] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:30.689796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ef59d-bfb9-4bc1-b00c-be8831abeb4b None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:30.699130 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7965118-1a09-49af-9998-23e8d6f2ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:30.700110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7965118-1a09-49af-9998-23e8d6f2ee40 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:30.700550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7965118-1a09-49af-9998-23e8d6f2ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:30.700827 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7965118-1a09-49af-9998-23e8d6f2ee40 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:30.701485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7965118-1a09-49af-9998-23e8d6f2ee40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:30.702910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7965118-1a09-49af-9998-23e8d6f2ee40 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:30.719045 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:30.733700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 365/706] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:15:30 2022] POST /identity/v3/users => generated 254 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:15:30.733700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaca980-4ee6-463b-ad66-4ef701036ba0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:30.733700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaca980-4ee6-463b-ad66-4ef701036ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:30.733700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaca980-4ee6-463b-ad66-4ef701036ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:30.749304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6eaca980-4ee6-463b-ad66-4ef701036ba0 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:30.750551 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6eaca980-4ee6-463b-ad66-4ef701036ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:30.836376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ef59d-bfb9-4bc1-b00c-be8831abeb4b None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:30.837411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 366/707] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:30.838346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaca980-4ee6-463b-ad66-4ef701036ba0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:30.876838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:30.888894 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8f4da-cd7f-4047-a002-0ee9f20c284f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:30.891368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8f4da-cd7f-4047-a002-0ee9f20c284f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:30.892018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8f4da-cd7f-4047-a002-0ee9f20c284f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:30.892502 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8f4da-cd7f-4047-a002-0ee9f20c284f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:30.894259 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8f4da-cd7f-4047-a002-0ee9f20c284f None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=f5fa28ffdafd435cb2d93bd453add3cb) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:30.894895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8f4da-cd7f-4047-a002-0ee9f20c284f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:30.897019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8f4da-cd7f-4047-a002-0ee9f20c284f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:30.904868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 342/708] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:30 2022] GET /identity/v3/role_assignments?user.id=f5fa28ffdafd435cb2d93bd453add3cb&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=23538dc4c770454e80dd11ed8e498539 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:30.982088 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaca980-4ee6-463b-ad66-4ef701036ba0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:31.005422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b755dc-a7dd-4511-a897-a6835798b233 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:31.005745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b755dc-a7dd-4511-a897-a6835798b233 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:31.005981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b755dc-a7dd-4511-a897-a6835798b233 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:31.012012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 343/709] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:31.013664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e28486-809e-4657-9c1c-8537842f0a15 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:31.013961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e28486-809e-4657-9c1c-8537842f0a15 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:31.014234 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e28486-809e-4657-9c1c-8537842f0a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:31.026667 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb7d9e-4978-425f-b723-f12e55e694a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:31.028490 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb7d9e-4978-425f-b723-f12e55e694a0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:31.028983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb7d9e-4978-425f-b723-f12e55e694a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:31.029635 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb7d9e-4978-425f-b723-f12e55e694a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:31.034120 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-d6e28486-809e-4657-9c1c-8537842f0a15 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:31.035431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d6e28486-809e-4657-9c1c-8537842f0a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:31.040176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb7d9e-4978-425f-b723-f12e55e694a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:31.041744 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb7d9e-4978-425f-b723-f12e55e694a0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f2cb7d9e-4978-425f-b723-f12e55e694a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:31.054658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:31.055936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:31.056829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:31.056829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:31.056829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:31.056829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:31.056829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:31.056829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:31.056829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:31.056829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:31.056829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 367/710] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:31.062374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb7d9e-4978-425f-b723-f12e55e694a0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:31.132927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 344/711] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:15:31.188665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:31.198303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb7d9e-4978-425f-b723-f12e55e694a0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:31.199864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e28486-809e-4657-9c1c-8537842f0a15 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:31.231940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:31.241386 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35430cd-757a-4a58-bf57-03baba752d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:31.242470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35430cd-757a-4a58-bf57-03baba752d4a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:31.242756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35430cd-757a-4a58-bf57-03baba752d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:31.243030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35430cd-757a-4a58-bf57-03baba752d4a None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:31.243768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35430cd-757a-4a58-bf57-03baba752d4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:31.244151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35430cd-757a-4a58-bf57-03baba752d4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:31.245587 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35430cd-757a-4a58-bf57-03baba752d4a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:31.259208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 368/712] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:31.266106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35430cd-757a-4a58-bf57-03baba752d4a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:31.344333 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e28486-809e-4657-9c1c-8537842f0a15 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:31.395993 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1604671-e145-4b67-952f-94ed3fa5f809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:31.397009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1604671-e145-4b67-952f-94ed3fa5f809 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:31.397485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1604671-e145-4b67-952f-94ed3fa5f809 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:31.398095 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1604671-e145-4b67-952f-94ed3fa5f809 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:31.398918 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1604671-e145-4b67-952f-94ed3fa5f809 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:31.400537 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1604671-e145-4b67-952f-94ed3fa5f809 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-f1604671-e145-4b67-952f-94ed3fa5f809 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:31.412179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:31.413287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:31.414758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:31.414758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:31.414758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:31.414758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:31.414758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:31.414758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:31.414758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:31.414758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:31.414758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:31.414758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:31.414758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:15:31.414758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:31.414758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:15:31.414758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:31.417621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 345/713] 10.222.0.27 () {56 vars in 1129 bytes} [Mon Sep 12 23:15:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:15:31.423591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1604671-e145-4b67-952f-94ed3fa5f809 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:31.438731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35430cd-757a-4a58-bf57-03baba752d4a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:31.482898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff0fca-c240-4525-8b0d-ac577b2bd3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:31.483883 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff0fca-c240-4525-8b0d-ac577b2bd3ca None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:31.484367 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff0fca-c240-4525-8b0d-ac577b2bd3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:31.484838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff0fca-c240-4525-8b0d-ac577b2bd3ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:31.485803 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff0fca-c240-4525-8b0d-ac577b2bd3ca None admin] RBAC: Adding query filter params (role.id=5c7e1f557ae04250a01db783f3c27a5f, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=c1f6bb09fd7f4905a1693ef7cb6f088e) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:31.486374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff0fca-c240-4525-8b0d-ac577b2bd3ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:31.488070 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff0fca-c240-4525-8b0d-ac577b2bd3ca None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:31.494987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 369/714] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:31 2022] GET /identity/v3/role_assignments?user.id=c1f6bb09fd7f4905a1693ef7cb6f088e&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=5c7e1f557ae04250a01db783f3c27a5f => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:31.496158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff0fca-c240-4525-8b0d-ac577b2bd3ca None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:31.533594 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:31.565023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:31.589651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1604671-e145-4b67-952f-94ed3fa5f809 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:31.668930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf4398-e36b-4dfa-83b8-a244d276ef5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:31.669950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf4398-e36b-4dfa-83b8-a244d276ef5c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:31.670428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf4398-e36b-4dfa-83b8-a244d276ef5c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:31.670918 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf4398-e36b-4dfa-83b8-a244d276ef5c None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:31.671789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf4398-e36b-4dfa-83b8-a244d276ef5c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:31.672628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf4398-e36b-4dfa-83b8-a244d276ef5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:31.676016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf4398-e36b-4dfa-83b8-a244d276ef5c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:31.693305 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 346/715] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Sep 12 23:15:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:31.694871 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf4398-e36b-4dfa-83b8-a244d276ef5c None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:31.702171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff0fca-c240-4525-8b0d-ac577b2bd3ca None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:31.747347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c2cc6-d12a-43b6-ae0e-0ad305dbc40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:31.748048 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c2cc6-d12a-43b6-ae0e-0ad305dbc40a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:31.748339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c2cc6-d12a-43b6-ae0e-0ad305dbc40a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:31.748618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c2cc6-d12a-43b6-ae0e-0ad305dbc40a None admin] PATH_INFO: `/v3/services/network` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:31.749276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c2cc6-d12a-43b6-ae0e-0ad305dbc40a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:31.751847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c2cc6-d12a-43b6-ae0e-0ad305dbc40a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-416c2cc6-d12a-43b6-ae0e-0ad305dbc40a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:31.772097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:31.773707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:31.774601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:31.774601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:31.774601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:31.774601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:31.774601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:31.774601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:31.774601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:31.774601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:31.774601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:31.774601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:31.774601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:15:31.774601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:31.774601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:15:31.774601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:31.777371 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 370/716] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:31 2022] GET /identity/v3/services/network => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:31.783552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c2cc6-d12a-43b6-ae0e-0ad305dbc40a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:31.825675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf4398-e36b-4dfa-83b8-a244d276ef5c None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:31.862075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28896a-840d-430d-a93f-c96beb645b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:31.862868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28896a-840d-430d-a93f-c96beb645b14 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:31.863240 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28896a-840d-430d-a93f-c96beb645b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:31.863633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28896a-840d-430d-a93f-c96beb645b14 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:31.864360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28896a-840d-430d-a93f-c96beb645b14 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:31.864759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28896a-840d-430d-a93f-c96beb645b14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:31.866222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28896a-840d-430d-a93f-c96beb645b14 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:31.879774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 347/717] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Sep 12 23:15:31 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:31.891100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:31.918520 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c2cc6-d12a-43b6-ae0e-0ad305dbc40a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:31.967698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b9407-22d4-4c4b-a587-816faa4ef55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:31.968323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b9407-22d4-4c4b-a587-816faa4ef55a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:31.968601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b9407-22d4-4c4b-a587-816faa4ef55a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:31.968871 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b9407-22d4-4c4b-a587-816faa4ef55a None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:31.969511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b9407-22d4-4c4b-a587-816faa4ef55a None admin] RBAC: Adding query filter params (name=network) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:31.969812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b9407-22d4-4c4b-a587-816faa4ef55a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:31.971214 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b9407-22d4-4c4b-a587-816faa4ef55a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:31.977434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 371/718] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Sep 12 23:15:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:31.982783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28896a-840d-430d-a93f-c96beb645b14 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:32.019185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:32.117408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28896a-840d-430d-a93f-c96beb645b14 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:32.153378 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3f1e7-8f65-40be-927f-39a29a6e0168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:32.153994 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3f1e7-8f65-40be-927f-39a29a6e0168 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:32.154266 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3f1e7-8f65-40be-927f-39a29a6e0168 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:32.154542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3f1e7-8f65-40be-927f-39a29a6e0168 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:32.155174 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3f1e7-8f65-40be-927f-39a29a6e0168 None admin] RBAC: Adding query filter params (type=network) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:32.155479 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3f1e7-8f65-40be-927f-39a29a6e0168 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:32.156887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3f1e7-8f65-40be-927f-39a29a6e0168 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:32.161998 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 348/719] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Sep 12 23:15:31 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:33.310165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5005d0f-2b2d-4b90-afae-08c09873d8d2 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:33.311152 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5005d0f-2b2d-4b90-afae-08c09873d8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:33.311665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5005d0f-2b2d-4b90-afae-08c09873d8d2 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:33.313474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 372/720] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:33.319534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d1fed-2ec6-4755-9e9f-e814cb5a8f88 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:33.319534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d1fed-2ec6-4755-9e9f-e814cb5a8f88 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:33.319534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d1fed-2ec6-4755-9e9f-e814cb5a8f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:33.344939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-356d1fed-2ec6-4755-9e9f-e814cb5a8f88 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:33.346433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-356d1fed-2ec6-4755-9e9f-e814cb5a8f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:33.438143 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 349/721] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:15:33.824933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb11ada-7041-4f45-b800-e96f5d889f1b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:33.825650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb11ada-7041-4f45-b800-e96f5d889f1b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:33.826153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb11ada-7041-4f45-b800-e96f5d889f1b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:33.827724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 373/722] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:33.840602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639144eb-efb5-4d07-a9eb-eab8e5268d61 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:33.840889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639144eb-efb5-4d07-a9eb-eab8e5268d61 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:33.841164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639144eb-efb5-4d07-a9eb-eab8e5268d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:33.846762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb11ada-7041-4f45-b800-e96f5d889f1b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:33.864209 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-639144eb-efb5-4d07-a9eb-eab8e5268d61 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:33.865416 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-639144eb-efb5-4d07-a9eb-eab8e5268d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:33.877520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:33.964680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb11ada-7041-4f45-b800-e96f5d889f1b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:34.003757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 350/723] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:34.009666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c471c62-bc76-450a-851e-42bf2d5270ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:34.010353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c471c62-bc76-450a-851e-42bf2d5270ef None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:34.010659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c471c62-bc76-450a-851e-42bf2d5270ef None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:34.010938 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c471c62-bc76-450a-851e-42bf2d5270ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:34.027859 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c471c62-bc76-450a-851e-42bf2d5270ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5c471c62-bc76-450a-851e-42bf2d5270ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:34.032986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:34.033913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:34.034760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:34.034760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:34.034760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:34.034760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:34.034760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:34.034760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:34.034760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:34.034760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:34.034760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:34.034760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:34.034760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:15:34.034760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:34.040238 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 374/724] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:15:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:34.046021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639144eb-efb5-4d07-a9eb-eab8e5268d61 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:34.208203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639144eb-efb5-4d07-a9eb-eab8e5268d61 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:34.260450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da419602-bf1c-43ea-8eb2-88f0dc91d509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:34.261071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419602-bf1c-43ea-8eb2-88f0dc91d509 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:34.261352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419602-bf1c-43ea-8eb2-88f0dc91d509 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:34.261623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419602-bf1c-43ea-8eb2-88f0dc91d509 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:34.262284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da419602-bf1c-43ea-8eb2-88f0dc91d509 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:34.262584 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da419602-bf1c-43ea-8eb2-88f0dc91d509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:34.263985 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da419602-bf1c-43ea-8eb2-88f0dc91d509 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:34.276593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 351/725] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:15:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:34.282302 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c471c62-bc76-450a-851e-42bf2d5270ef None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:34.291852 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:34.325556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48630da4-6b87-45f2-83c6-aa0f8d8067a8 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:34.326102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48630da4-6b87-45f2-83c6-aa0f8d8067a8 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:34.326102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48630da4-6b87-45f2-83c6-aa0f8d8067a8 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:34.327749 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 352/726] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:15:34.332401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff9b29-1408-4446-b90d-10e1ea871d4a None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:34.332679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff9b29-1408-4446-b90d-10e1ea871d4a None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:34.332946 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff9b29-1408-4446-b90d-10e1ea871d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:34.345158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:34.361104 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-d4ff9b29-1408-4446-b90d-10e1ea871d4a None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:34.362302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d4ff9b29-1408-4446-b90d-10e1ea871d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:34.409117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c471c62-bc76-450a-851e-42bf2d5270ef None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:34.447685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28d3e6-b148-4b23-88c5-776152190bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:34.448562 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28d3e6-b148-4b23-88c5-776152190bed None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:34.449018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28d3e6-b148-4b23-88c5-776152190bed None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:34.449440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28d3e6-b148-4b23-88c5-776152190bed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:34.461083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28d3e6-b148-4b23-88c5-776152190bed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:34.462717 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28d3e6-b148-4b23-88c5-776152190bed None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:34.465886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 353/727] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:15:34.467063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff9b29-1408-4446-b90d-10e1ea871d4a None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5d28d3e6-b148-4b23-88c5-776152190bed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:34.480730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:34.481916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:34.483065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:34.483065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:34.483065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:34.484606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 375/728] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:34.490247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28d3e6-b148-4b23-88c5-776152190bed None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:34.574709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:34.587190 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff9b29-1408-4446-b90d-10e1ea871d4a None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:34.642562 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b14b7-5ea2-47c9-aad9-40ca33363981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:34.643485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b14b7-5ea2-47c9-aad9-40ca33363981 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:34.643939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b14b7-5ea2-47c9-aad9-40ca33363981 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:34.644388 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b14b7-5ea2-47c9-aad9-40ca33363981 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:34.648280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:34.662475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b14b7-5ea2-47c9-aad9-40ca33363981 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-641b14b7-5ea2-47c9-aad9-40ca33363981 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:34.668352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:34.669578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:34.670638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:34.670638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:34.670638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:34.670638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:34.670638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:34.670638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:34.670638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:34.670638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:34.670638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:34.670638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:34.670638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:34.670638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:34.671426 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28d3e6-b148-4b23-88c5-776152190bed None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:34.673378 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 354/729] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:34.688423 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b14b7-5ea2-47c9-aad9-40ca33363981 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:34.721450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351166c-e3f8-4ff5-abe3-c9cbc6ac9700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:34.721993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351166c-e3f8-4ff5-abe3-c9cbc6ac9700 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:34.722276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351166c-e3f8-4ff5-abe3-c9cbc6ac9700 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:34.722552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351166c-e3f8-4ff5-abe3-c9cbc6ac9700 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:34.724502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351166c-e3f8-4ff5-abe3-c9cbc6ac9700 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:34.724823 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351166c-e3f8-4ff5-abe3-c9cbc6ac9700 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:34.726282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351166c-e3f8-4ff5-abe3-c9cbc6ac9700 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:34.754219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 376/730] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:34.758143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351166c-e3f8-4ff5-abe3-c9cbc6ac9700 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:34.856133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b14b7-5ea2-47c9-aad9-40ca33363981 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:34.888798 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351166c-e3f8-4ff5-abe3-c9cbc6ac9700 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:34.891969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cac390-3d5e-43f2-9fba-d402e7ec7c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:34.892812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cac390-3d5e-43f2-9fba-d402e7ec7c4a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:34.893274 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cac390-3d5e-43f2-9fba-d402e7ec7c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:34.893729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cac390-3d5e-43f2-9fba-d402e7ec7c4a None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:34.894584 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cac390-3d5e-43f2-9fba-d402e7ec7c4a None admin] RBAC: Adding query filter params (name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:34.895050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cac390-3d5e-43f2-9fba-d402e7ec7c4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:34.896464 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cac390-3d5e-43f2-9fba-d402e7ec7c4a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:34.906488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 355/731] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:34.908129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955582a1-d777-43f9-bde4-88751c07212a None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:34.908450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:34.908644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955582a1-d777-43f9-bde4-88751c07212a None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:34.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955582a1-d777-43f9-bde4-88751c07212a None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:34.913395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 356/732] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:34.913395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955582a1-d777-43f9-bde4-88751c07212a None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:34.931385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24df685-793d-4489-8a3d-2f94fdbe17dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:34.932060 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24df685-793d-4489-8a3d-2f94fdbe17dc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:34.932342 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24df685-793d-4489-8a3d-2f94fdbe17dc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:34.932612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24df685-793d-4489-8a3d-2f94fdbe17dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:34.939985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24df685-793d-4489-8a3d-2f94fdbe17dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:34.941428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24df685-793d-4489-8a3d-2f94fdbe17dc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b24df685-793d-4489-8a3d-2f94fdbe17dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:34.953260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:34.954398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:34.955369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:34.955369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:34.955369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:34.955369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:34.955369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:34.955369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:34.955369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:34.955369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:34.955369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:15:34.955369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:34.955369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 377/733] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Sep 12 23:15:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:34.956016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d3afe-714b-4d56-982d-f30654dfebf1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:34.956095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d3afe-714b-4d56-982d-f30654dfebf1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:34.956374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d3afe-714b-4d56-982d-f30654dfebf1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:34.958685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 378/734] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:34.959555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d3afe-714b-4d56-982d-f30654dfebf1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:34.973448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:35.018995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955582a1-d777-43f9-bde4-88751c07212a None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:35.052666 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332d05b-fd43-4ae8-af3f-62071a253a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:35.053485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332d05b-fd43-4ae8-af3f-62071a253a06 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:35.053773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332d05b-fd43-4ae8-af3f-62071a253a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:35.054049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332d05b-fd43-4ae8-af3f-62071a253a06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:35.061694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d3afe-714b-4d56-982d-f30654dfebf1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:35.063835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332d05b-fd43-4ae8-af3f-62071a253a06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-1332d05b-fd43-4ae8-af3f-62071a253a06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:35.069117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:35.070743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:35.072126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:35.072126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:35.072126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:35.072126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:35.072126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:35.072126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:35.072126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:35.072126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:35.072126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:35.072126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:35.072126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:15:35.072126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:35.072126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 357/735] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:15:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:35.072818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5cd53-50a8-4351-9b72-121f2009565d None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:35.072818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5cd53-50a8-4351-9b72-121f2009565d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:35.072818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5cd53-50a8-4351-9b72-121f2009565d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:35.090976 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-84c5cd53-50a8-4351-9b72-121f2009565d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:35.092207 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-84c5cd53-50a8-4351-9b72-121f2009565d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:35.096253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54d441-b968-4cdf-a1e0-bf1d71e3d88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:35.096881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54d441-b968-4cdf-a1e0-bf1d71e3d88b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:35.097162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54d441-b968-4cdf-a1e0-bf1d71e3d88b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:35.097437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54d441-b968-4cdf-a1e0-bf1d71e3d88b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:35.108770 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54d441-b968-4cdf-a1e0-bf1d71e3d88b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:35.110208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54d441-b968-4cdf-a1e0-bf1d71e3d88b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-db54d441-b968-4cdf-a1e0-bf1d71e3d88b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:35.124673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:35.125621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:35.126911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:35.126911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:35.126911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:35.126911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 379/736] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:35.126911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54d441-b968-4cdf-a1e0-bf1d71e3d88b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:35.154215 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:35.161317 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 358/737] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:35.162628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9d379-78f6-48c6-aa13-70fbc33eeca8 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:35.162908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9d379-78f6-48c6-aa13-70fbc33eeca8 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:35.163176 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9d379-78f6-48c6-aa13-70fbc33eeca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:35.186776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-2cc9d379-78f6-48c6-aa13-70fbc33eeca8 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:35.187995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2cc9d379-78f6-48c6-aa13-70fbc33eeca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:35.229580 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54d441-b968-4cdf-a1e0-bf1d71e3d88b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:35.232877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:35.258482 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 359/738] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:15:35.267748 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9d379-78f6-48c6-aa13-70fbc33eeca8 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:35.277194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ae3aa-d4ee-4e58-8bbd-98ad64a50288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:35.278270 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae3aa-d4ee-4e58-8bbd-98ad64a50288 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:35.278672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae3aa-d4ee-4e58-8bbd-98ad64a50288 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:35.279023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae3aa-d4ee-4e58-8bbd-98ad64a50288 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:35.279865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ae3aa-d4ee-4e58-8bbd-98ad64a50288 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:35.280339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ae3aa-d4ee-4e58-8bbd-98ad64a50288 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:35.282952 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ae3aa-d4ee-4e58-8bbd-98ad64a50288 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:35.303698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 380/739] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Sep 12 23:15:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:35.308565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ae3aa-d4ee-4e58-8bbd-98ad64a50288 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:35.371196 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9d379-78f6-48c6-aa13-70fbc33eeca8 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:35.407371 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:35.421738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020ee2f-e711-4733-a50f-a4d099c12bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:35.422590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020ee2f-e711-4733-a50f-a4d099c12bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:35.422873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020ee2f-e711-4733-a50f-a4d099c12bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:35.423145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020ee2f-e711-4733-a50f-a4d099c12bf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:35.424254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020ee2f-e711-4733-a50f-a4d099c12bf5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:35.424564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020ee2f-e711-4733-a50f-a4d099c12bf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:35.426236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020ee2f-e711-4733-a50f-a4d099c12bf5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:35.427535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ae3aa-d4ee-4e58-8bbd-98ad64a50288 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:35.436092 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 360/740] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:15:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:35.437418 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020ee2f-e711-4733-a50f-a4d099c12bf5 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:35.467778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3c758-1341-4489-a971-32cb67747e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:35.468710 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3c758-1341-4489-a971-32cb67747e6e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:35.469007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3c758-1341-4489-a971-32cb67747e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:35.469545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3c758-1341-4489-a971-32cb67747e6e None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:35.470237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3c758-1341-4489-a971-32cb67747e6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:35.470551 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3c758-1341-4489-a971-32cb67747e6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:35.472571 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3c758-1341-4489-a971-32cb67747e6e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:35.477363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:35.488009 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 381/741] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:35.488009 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3c758-1341-4489-a971-32cb67747e6e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:35.546140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020ee2f-e711-4733-a50f-a4d099c12bf5 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:35.582295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4b648-7018-4a70-b2d8-fb3539094ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:35.582930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4b648-7018-4a70-b2d8-fb3539094ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:35.583214 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4b648-7018-4a70-b2d8-fb3539094ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:35.583489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4b648-7018-4a70-b2d8-fb3539094ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:35.594309 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4b648-7018-4a70-b2d8-fb3539094ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:35.595784 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4b648-7018-4a70-b2d8-fb3539094ee5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:35.604484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3c758-1341-4489-a971-32cb67747e6e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-d6a4b648-7018-4a70-b2d8-fb3539094ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:35.609939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:35.611030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:35.612421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:35.612421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:35.612421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:35.612421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 361/742] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:35.612421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4b648-7018-4a70-b2d8-fb3539094ee5 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:35.639627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7438270-1183-4922-ab9c-0dd4249f4708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:35.640232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7438270-1183-4922-ab9c-0dd4249f4708 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:35.640510 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7438270-1183-4922-ab9c-0dd4249f4708 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:35.640788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7438270-1183-4922-ab9c-0dd4249f4708 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:35.641437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7438270-1183-4922-ab9c-0dd4249f4708 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:35.642803 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7438270-1183-4922-ab9c-0dd4249f4708 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:35.661547 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 382/743] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Sep 12 23:15:35 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:15:35.662361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7438270-1183-4922-ab9c-0dd4249f4708 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:35.677349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:35.711474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:35.723336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4b648-7018-4a70-b2d8-fb3539094ee5 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:35.767921 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73e016-22dd-4f73-a50b-c915d42ed979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:35.768468 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73e016-22dd-4f73-a50b-c915d42ed979 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:35.768757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73e016-22dd-4f73-a50b-c915d42ed979 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:35.769023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73e016-22dd-4f73-a50b-c915d42ed979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:35.781858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73e016-22dd-4f73-a50b-c915d42ed979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:35.783648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73e016-22dd-4f73-a50b-c915d42ed979 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:35.784811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7438270-1183-4922-ab9c-0dd4249f4708 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-cc73e016-22dd-4f73-a50b-c915d42ed979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:35.799509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:35.800556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:35.801440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:35.801440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:35.801440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:35.801440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 362/744] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:35.801661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73e016-22dd-4f73-a50b-c915d42ed979 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:35.827467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e72bab-cf23-4745-9218-15779f57487d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:35.828152 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e72bab-cf23-4745-9218-15779f57487d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:35.828433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e72bab-cf23-4745-9218-15779f57487d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:35.828716 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e72bab-cf23-4745-9218-15779f57487d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:35.837659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e72bab-cf23-4745-9218-15779f57487d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:35.839110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e72bab-cf23-4745-9218-15779f57487d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f8e72bab-cf23-4745-9218-15779f57487d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:35.853844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:35.854875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:35.856003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:35.856003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:35.856003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:35.856003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:35.856003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:35.856003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:35.856003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:35.856003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:35.856003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:15:35.856003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:35.856003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 383/745] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Sep 12 23:15:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:35.856003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e72bab-cf23-4745-9218-15779f57487d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:35.924094 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73e016-22dd-4f73-a50b-c915d42ed979 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:35.938164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:35.961669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121bdda-2796-4e0a-b793-816260f03e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:35.962488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121bdda-2796-4e0a-b793-816260f03e54 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:35.962601 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121bdda-2796-4e0a-b793-816260f03e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:35.962814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121bdda-2796-4e0a-b793-816260f03e54 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:35.963450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9121bdda-2796-4e0a-b793-816260f03e54 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:35.964863 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9121bdda-2796-4e0a-b793-816260f03e54 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:35.975155 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e72bab-cf23-4745-9218-15779f57487d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:35.976746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:35.992107 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 363/746] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Sep 12 23:15:35 2022] POST /identity/v3/services => generated 240 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:15:35.993151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121bdda-2796-4e0a-b793-816260f03e54 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:36.037139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8cb7d-ebb5-464f-916a-f2362db590f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:36.037471 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8cb7d-ebb5-464f-916a-f2362db590f5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:36.037767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8cb7d-ebb5-464f-916a-f2362db590f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:36.038093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8cb7d-ebb5-464f-916a-f2362db590f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:36.039368 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8cb7d-ebb5-464f-916a-f2362db590f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:36.039741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8cb7d-ebb5-464f-916a-f2362db590f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:36.041241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8cb7d-ebb5-464f-916a-f2362db590f5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:36.055999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 384/747] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:36.057091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8cb7d-ebb5-464f-916a-f2362db590f5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:36.113926 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121bdda-2796-4e0a-b793-816260f03e54 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:36.153647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb7b18-9dc7-438c-a960-6b3b13687c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:36.154638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb7b18-9dc7-438c-a960-6b3b13687c19 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:36.155137 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb7b18-9dc7-438c-a960-6b3b13687c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:36.156234 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb7b18-9dc7-438c-a960-6b3b13687c19 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:36.157098 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb7b18-9dc7-438c-a960-6b3b13687c19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:36.157882 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb7b18-9dc7-438c-a960-6b3b13687c19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:36.159461 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb7b18-9dc7-438c-a960-6b3b13687c19 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:36.172384 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 364/748] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:36.173761 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb7b18-9dc7-438c-a960-6b3b13687c19 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:36.177716 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8cb7d-ebb5-464f-916a-f2362db590f5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:36.205671 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:36.218953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb521869-f1e3-4c3f-b11e-8aff85a0b064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:36.219597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb521869-f1e3-4c3f-b11e-8aff85a0b064 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:36.219872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb521869-f1e3-4c3f-b11e-8aff85a0b064 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:36.220146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb521869-f1e3-4c3f-b11e-8aff85a0b064 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:36.220875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb521869-f1e3-4c3f-b11e-8aff85a0b064 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:36.221184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb521869-f1e3-4c3f-b11e-8aff85a0b064 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:36.222683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb521869-f1e3-4c3f-b11e-8aff85a0b064 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:36.242829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 385/749] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Sep 12 23:15:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:36.244591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb521869-f1e3-4c3f-b11e-8aff85a0b064 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:36.251066 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:36.305997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb7b18-9dc7-438c-a960-6b3b13687c19 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:36.343259 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686ce3d-f234-4792-83b3-dc80a4c2f5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:36.343934 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686ce3d-f234-4792-83b3-dc80a4c2f5a7 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:36.344219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686ce3d-f234-4792-83b3-dc80a4c2f5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:36.344753 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686ce3d-f234-4792-83b3-dc80a4c2f5a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:36.354830 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686ce3d-f234-4792-83b3-dc80a4c2f5a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:36.356198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb521869-f1e3-4c3f-b11e-8aff85a0b064 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:36.356339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686ce3d-f234-4792-83b3-dc80a4c2f5a7 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-5686ce3d-f234-4792-83b3-dc80a4c2f5a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:36.371491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:36.372964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:36.373889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:36.373889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:36.373889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:36.373889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:36.373889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:36.373889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:36.373889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:36.373889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:36.373889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 365/750] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:36.373889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686ce3d-f234-4792-83b3-dc80a4c2f5a7 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:36.400511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac319f7e-a7e6-489d-8ddc-98964141259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:36.401152 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac319f7e-a7e6-489d-8ddc-98964141259e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:36.401444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac319f7e-a7e6-489d-8ddc-98964141259e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:36.401730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac319f7e-a7e6-489d-8ddc-98964141259e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:36.411281 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac319f7e-a7e6-489d-8ddc-98964141259e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:36.412829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac319f7e-a7e6-489d-8ddc-98964141259e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ac319f7e-a7e6-489d-8ddc-98964141259e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:36.428582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:36.430544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:36.431474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:36.431474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:36.431474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:36.431474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:36.431474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:36.431474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:36.431474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:36.431474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:36.431474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:15:36.431474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:36.431474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 386/751] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:15:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:36.432093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac319f7e-a7e6-489d-8ddc-98964141259e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:36.481652 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:36.491822 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686ce3d-f234-4792-83b3-dc80a4c2f5a7 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:36.523984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:36.540494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131091f1-f5aa-492c-98f4-87192838729f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:36.541693 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131091f1-f5aa-492c-98f4-87192838729f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:36.542205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131091f1-f5aa-492c-98f4-87192838729f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:36.542806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131091f1-f5aa-492c-98f4-87192838729f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:36.553208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac319f7e-a7e6-489d-8ddc-98964141259e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:36.594430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc34683-f653-4285-b595-2ad170287b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:36.595033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc34683-f653-4285-b595-2ad170287b32 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:36.595364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc34683-f653-4285-b595-2ad170287b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:36.595706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc34683-f653-4285-b595-2ad170287b32 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:36.596434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc34683-f653-4285-b595-2ad170287b32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:36.596746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc34683-f653-4285-b595-2ad170287b32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:36.598918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc34683-f653-4285-b595-2ad170287b32 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:36.618090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 387/752] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:36.619394 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc34683-f653-4285-b595-2ad170287b32 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:36.638598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131091f1-f5aa-492c-98f4-87192838729f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:36.640103 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131091f1-f5aa-492c-98f4-87192838729f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:36.736660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:36.741513 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc34683-f653-4285-b595-2ad170287b32 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:36.754431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 366/753] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:15:36.755511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131091f1-f5aa-492c-98f4-87192838729f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:36.775913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:36.787605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb6b7e-175e-462a-bd31-c16f72c0c9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:36.789023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb6b7e-175e-462a-bd31-c16f72c0c9dc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:36.789528 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb6b7e-175e-462a-bd31-c16f72c0c9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:36.790244 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb6b7e-175e-462a-bd31-c16f72c0c9dc None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:36.791871 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb6b7e-175e-462a-bd31-c16f72c0c9dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:36.792386 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb6b7e-175e-462a-bd31-c16f72c0c9dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:36.794123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb6b7e-175e-462a-bd31-c16f72c0c9dc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:36.813958 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 388/754] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Sep 12 23:15:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:36.815217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb6b7e-175e-462a-bd31-c16f72c0c9dc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:36.879801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131091f1-f5aa-492c-98f4-87192838729f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:36.919893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef82ba-1236-4eca-a835-d98a5aa47dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:36.920564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef82ba-1236-4eca-a835-d98a5aa47dc3 None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:36.920848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef82ba-1236-4eca-a835-d98a5aa47dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:36.921128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef82ba-1236-4eca-a835-d98a5aa47dc3 None admin] PATH_INFO: `/v3/projects/32bbb936501846e8914acf07890c2891/users/f5fa28ffdafd435cb2d93bd453add3cb/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:36.923142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef82ba-1236-4eca-a835-d98a5aa47dc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=32bbb936501846e8914acf07890c2891, user_id=f5fa28ffdafd435cb2d93bd453add3cb, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:36.924691 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef82ba-1236-4eca-a835-d98a5aa47dc3 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:36.947449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb6b7e-175e-462a-bd31-c16f72c0c9dc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:36.947840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 367/755] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:15:36 2022] PUT /identity/v3/projects/32bbb936501846e8914acf07890c2891/users/f5fa28ffdafd435cb2d93bd453add3cb/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:15:36.948975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef82ba-1236-4eca-a835-d98a5aa47dc3 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:36.989294 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a62690-abaa-4b2c-861d-728b8cffba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:36.990331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a62690-abaa-4b2c-861d-728b8cffba01 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:36.990837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a62690-abaa-4b2c-861d-728b8cffba01 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:36.991407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a62690-abaa-4b2c-861d-728b8cffba01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:37.003971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a62690-abaa-4b2c-861d-728b8cffba01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:37.005820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a62690-abaa-4b2c-861d-728b8cffba01 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-84a62690-abaa-4b2c-861d-728b8cffba01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:37.022242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:37.024160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:37.025057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:37.025057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:37.025057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:37.031500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 389/756] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:37.033613 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a62690-abaa-4b2c-861d-728b8cffba01 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:37.047283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:37.101272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:37.143724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef82ba-1236-4eca-a835-d98a5aa47dc3 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:37.146304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a62690-abaa-4b2c-861d-728b8cffba01 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:37.183819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b12ce-c52d-49a9-a784-55a8fb16866c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:37.184842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b12ce-c52d-49a9-a784-55a8fb16866c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:37.185347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b12ce-c52d-49a9-a784-55a8fb16866c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:37.185844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b12ce-c52d-49a9-a784-55a8fb16866c None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:37.186697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b12ce-c52d-49a9-a784-55a8fb16866c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:37.187201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b12ce-c52d-49a9-a784-55a8fb16866c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:37.189097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b12ce-c52d-49a9-a784-55a8fb16866c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:37.208017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 390/757] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:37.210203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b12ce-c52d-49a9-a784-55a8fb16866c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:37.214515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceaa569-37c6-432b-838b-9c7094484f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:37.215930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceaa569-37c6-432b-838b-9c7094484f77 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:37.215930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceaa569-37c6-432b-838b-9c7094484f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:37.215930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceaa569-37c6-432b-838b-9c7094484f77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:37.287033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:37.321849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceaa569-37c6-432b-838b-9c7094484f77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:37.323442 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceaa569-37c6-432b-838b-9c7094484f77 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:37.365765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b12ce-c52d-49a9-a784-55a8fb16866c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:37.409395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:37.416465 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b4736-d7d9-48f4-888d-7423c91d2b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:37.417359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b4736-d7d9-48f4-888d-7423c91d2b53 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:37.418003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b4736-d7d9-48f4-888d-7423c91d2b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:37.418350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b4736-d7d9-48f4-888d-7423c91d2b53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:37.427683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b4736-d7d9-48f4-888d-7423c91d2b53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:37.429899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b4736-d7d9-48f4-888d-7423c91d2b53 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:37.434955 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 368/758] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:15:37.440568 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceaa569-37c6-432b-838b-9c7094484f77 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a70b4736-d7d9-48f4-888d-7423c91d2b53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:37.445416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:37.446299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:37.447366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:37.447366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:37.447366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:37.447366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:37.447366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:37.447366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:37.447366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:37.447366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:37.447366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 391/759] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:15:37.453937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b4736-d7d9-48f4-888d-7423c91d2b53 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:37.559010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceaa569-37c6-432b-838b-9c7094484f77 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:37.598143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b4736-d7d9-48f4-888d-7423c91d2b53 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:37.606156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b91e5f-a2cd-46e4-8637-471437ce62d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:37.607019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b91e5f-a2cd-46e4-8637-471437ce62d4 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:37.607476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b91e5f-a2cd-46e4-8637-471437ce62d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:37.607971 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b91e5f-a2cd-46e4-8637-471437ce62d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:37.620631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b91e5f-a2cd-46e4-8637-471437ce62d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:37.622647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b91e5f-a2cd-46e4-8637-471437ce62d4 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-18b91e5f-a2cd-46e4-8637-471437ce62d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:37.640022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:37.641006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:37.641884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:37.641884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:37.641884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:37.644342 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 369/760] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:15:37.650787 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b91e5f-a2cd-46e4-8637-471437ce62d4 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:37.661377 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:37.672999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf94ea4-0b10-479f-8629-62b3889bd4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:37.673622 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf94ea4-0b10-479f-8629-62b3889bd4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:37.673906 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf94ea4-0b10-479f-8629-62b3889bd4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:37.674183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf94ea4-0b10-479f-8629-62b3889bd4d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:37.674872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf94ea4-0b10-479f-8629-62b3889bd4d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:37.675178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf94ea4-0b10-479f-8629-62b3889bd4d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:37.676745 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf94ea4-0b10-479f-8629-62b3889bd4d3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:37.696374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:37.704986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 392/761] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:37.711620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf94ea4-0b10-479f-8629-62b3889bd4d3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:37.791247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b91e5f-a2cd-46e4-8637-471437ce62d4 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:37.847922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135fc00-9cea-4ea8-9b98-be84c630ed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:37.848863 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135fc00-9cea-4ea8-9b98-be84c630ed31 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:37.849149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135fc00-9cea-4ea8-9b98-be84c630ed31 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:37.849481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135fc00-9cea-4ea8-9b98-be84c630ed31 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:37.850130 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135fc00-9cea-4ea8-9b98-be84c630ed31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:37.850536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135fc00-9cea-4ea8-9b98-be84c630ed31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:37.852483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135fc00-9cea-4ea8-9b98-be84c630ed31 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:37.866181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 370/762] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:37.872095 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135fc00-9cea-4ea8-9b98-be84c630ed31 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:37.893728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf94ea4-0b10-479f-8629-62b3889bd4d3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:37.984900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:37.998259 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135fc00-9cea-4ea8-9b98-be84c630ed31 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:38.005218 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb5ad6-d353-4a89-b51b-fa1c91417114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:38.005839 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb5ad6-d353-4a89-b51b-fa1c91417114 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:38.006119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb5ad6-d353-4a89-b51b-fa1c91417114 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:38.006390 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb5ad6-d353-4a89-b51b-fa1c91417114 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:38.007114 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb5ad6-d353-4a89-b51b-fa1c91417114 None admin] RBAC: Adding query filter params (role.id=5c7e1f557ae04250a01db783f3c27a5f, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=0212eb5841a04075a0fd7ba853231267) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:38.007540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb5ad6-d353-4a89-b51b-fa1c91417114 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:38.008983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb5ad6-d353-4a89-b51b-fa1c91417114 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:38.019979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 393/763] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:37 2022] GET /identity/v3/role_assignments?user.id=0212eb5841a04075a0fd7ba853231267&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=5c7e1f557ae04250a01db783f3c27a5f => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:38.048016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a5d7f-15b0-40a1-bbb1-ef1acc68df32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:38.048672 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a5d7f-15b0-40a1-bbb1-ef1acc68df32 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:38.048953 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a5d7f-15b0-40a1-bbb1-ef1acc68df32 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:38.049229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a5d7f-15b0-40a1-bbb1-ef1acc68df32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:38.061658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a5d7f-15b0-40a1-bbb1-ef1acc68df32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:38.063171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a5d7f-15b0-40a1-bbb1-ef1acc68df32 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-cb1a5d7f-15b0-40a1-bbb1-ef1acc68df32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:38.081245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:38.082206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:38.083109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:38.083109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:38.083109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:38.083109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:38.083109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:38.083109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:38.083109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:38.083109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:38.085068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 371/764] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:38.090248 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb5ad6-d353-4a89-b51b-fa1c91417114 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:38.106950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20425d2-0c7d-4bad-a2c7-9d590e274b00 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:38.106950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20425d2-0c7d-4bad-a2c7-9d590e274b00 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:38.106950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20425d2-0c7d-4bad-a2c7-9d590e274b00 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:38.107714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 372/765] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:38.117961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1dc40-0d1d-4b88-b4c8-4bc37e3287d4 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:38.118580 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1dc40-0d1d-4b88-b4c8-4bc37e3287d4 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:38.119009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1dc40-0d1d-4b88-b4c8-4bc37e3287d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:38.153100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-9fc1dc40-0d1d-4b88-b4c8-4bc37e3287d4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:38.154811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9fc1dc40-0d1d-4b88-b4c8-4bc37e3287d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:38.170618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:38.243630 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb5ad6-d353-4a89-b51b-fa1c91417114 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:38.298137 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 373/766] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:15:38.318566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb52d0e-4fef-481f-b565-ad02f42361ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:38.319202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb52d0e-4fef-481f-b565-ad02f42361ab None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:38.319484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb52d0e-4fef-481f-b565-ad02f42361ab None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:38.320316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb52d0e-4fef-481f-b565-ad02f42361ab None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:38.321110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb52d0e-4fef-481f-b565-ad02f42361ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:38.321460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb52d0e-4fef-481f-b565-ad02f42361ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:38.322968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb52d0e-4fef-481f-b565-ad02f42361ab None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:38.341913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 394/767] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:38.348036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1dc40-0d1d-4b88-b4c8-4bc37e3287d4 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:38.399679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d2321-f339-43df-85be-ea83fb20d9e9 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:38.399992 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d2321-f339-43df-85be-ea83fb20d9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:38.400242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d2321-f339-43df-85be-ea83fb20d9e9 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:38.401589 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 395/768] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:15:38.414417 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a640b4e-d4cb-4634-9ba1-34042eb2fd8f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:38.414691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a640b4e-d4cb-4634-9ba1-34042eb2fd8f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:38.414974 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a640b4e-d4cb-4634-9ba1-34042eb2fd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:38.452549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4a640b4e-d4cb-4634-9ba1-34042eb2fd8f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:38.453772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4a640b4e-d4cb-4634-9ba1-34042eb2fd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:38.458183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:38.490584 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1dc40-0d1d-4b88-b4c8-4bc37e3287d4 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:38.541055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 396/769] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:15:38.559137 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e368e-0a8c-4e50-a694-9846838edcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:38.559784 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e368e-0a8c-4e50-a694-9846838edcfe None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:38.560070 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e368e-0a8c-4e50-a694-9846838edcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:38.560348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e368e-0a8c-4e50-a694-9846838edcfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:38.561081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e368e-0a8c-4e50-a694-9846838edcfe None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=c1f6bb09fd7f4905a1693ef7cb6f088e) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:38.561394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e368e-0a8c-4e50-a694-9846838edcfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:38.562981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e368e-0a8c-4e50-a694-9846838edcfe None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:38.569666 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 374/770] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:38 2022] GET /identity/v3/role_assignments?user.id=c1f6bb09fd7f4905a1693ef7cb6f088e&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=23538dc4c770454e80dd11ed8e498539 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:38.597373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a640b4e-d4cb-4634-9ba1-34042eb2fd8f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:38.654962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:38.730949 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a640b4e-d4cb-4634-9ba1-34042eb2fd8f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:38.769971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a199153-73d3-4107-baa7-63d1c82a8d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:38.770608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a199153-73d3-4107-baa7-63d1c82a8d19 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:38.770889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a199153-73d3-4107-baa7-63d1c82a8d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:38.771162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a199153-73d3-4107-baa7-63d1c82a8d19 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:38.771782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a199153-73d3-4107-baa7-63d1c82a8d19 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:38.773190 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a199153-73d3-4107-baa7-63d1c82a8d19 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-1a199153-73d3-4107-baa7-63d1c82a8d19 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:38.782338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:38.783631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:38.784477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 397/771] 10.222.0.27 () {56 vars in 1129 bytes} [Mon Sep 12 23:15:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:38.791815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e368e-0a8c-4e50-a694-9846838edcfe None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:38.890307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a199153-73d3-4107-baa7-63d1c82a8d19 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:38.972922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e368e-0a8c-4e50-a694-9846838edcfe None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:39.043002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf6be5-9af3-44c7-b307-cd4cc4eb7251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:39.043542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf6be5-9af3-44c7-b307-cd4cc4eb7251 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:39.043850 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf6be5-9af3-44c7-b307-cd4cc4eb7251 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:39.044128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf6be5-9af3-44c7-b307-cd4cc4eb7251 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:39.044759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf6be5-9af3-44c7-b307-cd4cc4eb7251 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:39.045066 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf6be5-9af3-44c7-b307-cd4cc4eb7251 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:39.046444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf6be5-9af3-44c7-b307-cd4cc4eb7251 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:39.057598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 375/772] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Sep 12 23:15:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:39.064695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf6be5-9af3-44c7-b307-cd4cc4eb7251 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:39.078034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a199153-73d3-4107-baa7-63d1c82a8d19 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:39.079084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:39.112653 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3aa944-93ed-4e61-82af-e57ee4ad90f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:39.113280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3aa944-93ed-4e61-82af-e57ee4ad90f9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:39.113557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3aa944-93ed-4e61-82af-e57ee4ad90f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:39.113830 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3aa944-93ed-4e61-82af-e57ee4ad90f9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:39.114413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3aa944-93ed-4e61-82af-e57ee4ad90f9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:39.115804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3aa944-93ed-4e61-82af-e57ee4ad90f9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:39.119979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6b3aa944-93ed-4e61-82af-e57ee4ad90f9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:39.126269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:39.127496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:39.128590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:39.128590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:39.128590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:39.128590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:39.128590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:39.128590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:39.128590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:39.128590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:39.128590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:39.128590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:39.128590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:15:39.128590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:39.128590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:15:39.128590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:39.129792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 398/773] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:38 2022] GET /identity/v3/services/network => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:15:39.140062 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3aa944-93ed-4e61-82af-e57ee4ad90f9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:39.213135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf6be5-9af3-44c7-b307-cd4cc4eb7251 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:39.257082 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6867af-8ab1-4aea-91b4-26f21782ada4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:39.257739 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6867af-8ab1-4aea-91b4-26f21782ada4 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:39.258022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6867af-8ab1-4aea-91b4-26f21782ada4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:39.258301 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6867af-8ab1-4aea-91b4-26f21782ada4 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:39.258937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6867af-8ab1-4aea-91b4-26f21782ada4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:39.259244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6867af-8ab1-4aea-91b4-26f21782ada4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:39.260653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6867af-8ab1-4aea-91b4-26f21782ada4 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:39.266132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 376/774] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Sep 12 23:15:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:39.272350 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6867af-8ab1-4aea-91b4-26f21782ada4 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:39.325088 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3aa944-93ed-4e61-82af-e57ee4ad90f9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:39.371822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce1d42-2add-4956-bd23-53998732b97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:39.372606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce1d42-2add-4956-bd23-53998732b97b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:39.372885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce1d42-2add-4956-bd23-53998732b97b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:39.373160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce1d42-2add-4956-bd23-53998732b97b None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:39.373836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce1d42-2add-4956-bd23-53998732b97b None admin] RBAC: Adding query filter params (name=network) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:39.374139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce1d42-2add-4956-bd23-53998732b97b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:39.375494 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce1d42-2add-4956-bd23-53998732b97b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:39.380433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 399/775] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Sep 12 23:15:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:39.385764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce1d42-2add-4956-bd23-53998732b97b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:39.419729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:39.430051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:39.474222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6867af-8ab1-4aea-91b4-26f21782ada4 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:39.503935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce1d42-2add-4956-bd23-53998732b97b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:39.549408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176b66b-1f12-42e5-902b-1b609c4dbfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:39.550729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176b66b-1f12-42e5-902b-1b609c4dbfcb None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:39.551004 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176b66b-1f12-42e5-902b-1b609c4dbfcb None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:39.551275 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176b66b-1f12-42e5-902b-1b609c4dbfcb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:39.551995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176b66b-1f12-42e5-902b-1b609c4dbfcb None admin] RBAC: Adding query filter params (interface=public, service_id=f0f1a670f801470094572670e010185a, region_id=RegionOne) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:39.552302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176b66b-1f12-42e5-902b-1b609c4dbfcb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:39.553669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176b66b-1f12-42e5-902b-1b609c4dbfcb None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:39.565376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edde8c-9c8c-49d5-8eeb-de33610753a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:39.566041 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edde8c-9c8c-49d5-8eeb-de33610753a1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:39.566444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edde8c-9c8c-49d5-8eeb-de33610753a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:39.566718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edde8c-9c8c-49d5-8eeb-de33610753a1 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:39.567343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edde8c-9c8c-49d5-8eeb-de33610753a1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:39.567664 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edde8c-9c8c-49d5-8eeb-de33610753a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:39.569082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edde8c-9c8c-49d5-8eeb-de33610753a1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:39.573822 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 377/776] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Sep 12 23:15:39 2022] GET /identity/v3/endpoints?service_id=f0f1a670f801470094572670e010185a&interface=public®ion_id=RegionOne => generated 194 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:39.581297 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176b66b-1f12-42e5-902b-1b609c4dbfcb None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:39.587728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 400/777] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Sep 12 23:15:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:39.593209 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edde8c-9c8c-49d5-8eeb-de33610753a1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:39.689908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176b66b-1f12-42e5-902b-1b609c4dbfcb None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:39.728970 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f59eb6-9e19-426b-b13e-f15a0cf71f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:39.729599 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f59eb6-9e19-426b-b13e-f15a0cf71f2f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:39.729878 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f59eb6-9e19-426b-b13e-f15a0cf71f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:39.730156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f59eb6-9e19-426b-b13e-f15a0cf71f2f None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:39.730772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f59eb6-9e19-426b-b13e-f15a0cf71f2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:39.732273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f59eb6-9e19-426b-b13e-f15a0cf71f2f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:39.734383 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:39.744028 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 378/778] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Sep 12 23:15:39 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:39.817843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:39.821932 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edde8c-9c8c-49d5-8eeb-de33610753a1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:39.859958 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174e99b-2231-4fde-9e2f-2aad1be846b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:39.860498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174e99b-2231-4fde-9e2f-2aad1be846b3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:39.860781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174e99b-2231-4fde-9e2f-2aad1be846b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:39.861056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174e99b-2231-4fde-9e2f-2aad1be846b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:39.861769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174e99b-2231-4fde-9e2f-2aad1be846b3 None admin] RBAC: Adding query filter params (interface=public, service_id=7987e3fbad4746278b2a71ad2351850f, region_id=RegionOne) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:39.862083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174e99b-2231-4fde-9e2f-2aad1be846b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:39.863493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174e99b-2231-4fde-9e2f-2aad1be846b3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:39.872454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 401/779] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Sep 12 23:15:39 2022] GET /identity/v3/endpoints?service_id=7987e3fbad4746278b2a71ad2351850f&interface=public®ion_id=RegionOne => generated 194 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:39.877991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f59eb6-9e19-426b-b13e-f15a0cf71f2f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:39.905851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048703a-5f32-4178-9c67-1fe370548b75 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:39.906163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048703a-5f32-4178-9c67-1fe370548b75 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:39.906419 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048703a-5f32-4178-9c67-1fe370548b75 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:39.908338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 402/780] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:15:39.925677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae76fd-00ea-494a-bd65-08c97f435007 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:39.925977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae76fd-00ea-494a-bd65-08c97f435007 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:39.926232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae76fd-00ea-494a-bd65-08c97f435007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:39.954075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-26ae76fd-00ea-494a-bd65-08c97f435007 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:39.955361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-26ae76fd-00ea-494a-bd65-08c97f435007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:40.016837 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f59eb6-9e19-426b-b13e-f15a0cf71f2f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:40.063891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22243e-22f1-4a41-a553-e87325afe333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:40.064522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22243e-22f1-4a41-a553-e87325afe333 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:40.064804 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22243e-22f1-4a41-a553-e87325afe333 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:40.065130 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22243e-22f1-4a41-a553-e87325afe333 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:40.065756 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22243e-22f1-4a41-a553-e87325afe333 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:40.067159 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22243e-22f1-4a41-a553-e87325afe333 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:40.075139 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 379/781] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Sep 12 23:15:39 2022] GET /identity/v3/services => generated 772 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:40.158047 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 403/782] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:15:40.533417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22243e-22f1-4a41-a553-e87325afe333 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:40.608818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:40.669661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22243e-22f1-4a41-a553-e87325afe333 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:40.705495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e67029-3f6d-41e3-9f46-8d6ddfeb7495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:40.706500 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e67029-3f6d-41e3-9f46-8d6ddfeb7495 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:40.706975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e67029-3f6d-41e3-9f46-8d6ddfeb7495 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:40.707437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e67029-3f6d-41e3-9f46-8d6ddfeb7495 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:40.718545 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e67029-3f6d-41e3-9f46-8d6ddfeb7495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-50e67029-3f6d-41e3-9f46-8d6ddfeb7495 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:40.724228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:40.725147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:40.726063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:40.726063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:40.726063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:40.726063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:40.726063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:40.726063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:40.726063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:40.726063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:40.726063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:40.726063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:40.726063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:15:40.726063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:40.728656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 380/783] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:15:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:40.735628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae76fd-00ea-494a-bd65-08c97f435007 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:40.788585 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991af26-f6d3-408d-a080-ea16df53da38 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:40.789185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991af26-f6d3-408d-a080-ea16df53da38 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:40.789654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991af26-f6d3-408d-a080-ea16df53da38 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:40.791419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 381/784] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:40.796677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8234d-4951-49b0-8476-6d19e9087452 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:40.797193 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8234d-4951-49b0-8476-6d19e9087452 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:40.797660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8234d-4951-49b0-8476-6d19e9087452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:40.816489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c2b8234d-4951-49b0-8476-6d19e9087452 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:40.818052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c2b8234d-4951-49b0-8476-6d19e9087452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:40.870593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:40.882453 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae76fd-00ea-494a-bd65-08c97f435007 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:40.936709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 382/785] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:15:40.962661 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64848b3-4900-44ee-8340-b5393be5a4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:40.963663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64848b3-4900-44ee-8340-b5393be5a4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:40.964176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64848b3-4900-44ee-8340-b5393be5a4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:40.964685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64848b3-4900-44ee-8340-b5393be5a4b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:40.965565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64848b3-4900-44ee-8340-b5393be5a4b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:40.966077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64848b3-4900-44ee-8340-b5393be5a4b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:40.967666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64848b3-4900-44ee-8340-b5393be5a4b5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:40.979804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 404/786] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:15:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:40.985489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8234d-4951-49b0-8476-6d19e9087452 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:41.052425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:41.109475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8234d-4951-49b0-8476-6d19e9087452 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:41.159898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ecfde-2c6f-4043-b768-21adf1eb7970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:41.160631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ecfde-2c6f-4043-b768-21adf1eb7970 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:41.160916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ecfde-2c6f-4043-b768-21adf1eb7970 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:41.161187 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ecfde-2c6f-4043-b768-21adf1eb7970 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:41.171890 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ecfde-2c6f-4043-b768-21adf1eb7970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:41.177157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ecfde-2c6f-4043-b768-21adf1eb7970 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-da1ecfde-2c6f-4043-b768-21adf1eb7970 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:41.191917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:41.192853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:41.193747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:41.193747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:41.193747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:41.193747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 383/787] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:41.204391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64848b3-4900-44ee-8340-b5393be5a4b5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:41.248660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ecfde-2c6f-4043-b768-21adf1eb7970 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:41.390864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64848b3-4900-44ee-8340-b5393be5a4b5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:41.406199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ecfde-2c6f-4043-b768-21adf1eb7970 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:41.426497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e0350-aa95-4514-afbe-660fb11cba6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:41.427183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e0350-aa95-4514-afbe-660fb11cba6c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:41.427642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e0350-aa95-4514-afbe-660fb11cba6c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:41.427915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:41.428019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e0350-aa95-4514-afbe-660fb11cba6c None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:41.428684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e0350-aa95-4514-afbe-660fb11cba6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:41.429560 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e0350-aa95-4514-afbe-660fb11cba6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:41.430972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e0350-aa95-4514-afbe-660fb11cba6c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:41.443281 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 405/788] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:41.445249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773fb81-f786-470c-986b-eaba9e39154d None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:41.445621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773fb81-f786-470c-986b-eaba9e39154d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:41.445963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773fb81-f786-470c-986b-eaba9e39154d None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:41.450467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c2406-c8c6-4f3b-a2f5-8971738e2608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:41.451434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 406/789] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:41.451818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c2406-c8c6-4f3b-a2f5-8971738e2608 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:41.452249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773fb81-f786-470c-986b-eaba9e39154d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:41.452494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c2406-c8c6-4f3b-a2f5-8971738e2608 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:41.453027 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c2406-c8c6-4f3b-a2f5-8971738e2608 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:41.466886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c2406-c8c6-4f3b-a2f5-8971738e2608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-c42c2406-c8c6-4f3b-a2f5-8971738e2608 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:41.473854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:41.474796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:41.475683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:41.475683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:41.475683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:41.475683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:41.475683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:41.475683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:41.475683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:41.475683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:41.475683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:41.475683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:41.475683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:41.475683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:41.479663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 384/790] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:41.481777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d9b2d-6d68-4342-8a46-002325c5c7f4 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:41.482610 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:41.483087 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d9b2d-6d68-4342-8a46-002325c5c7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:41.485064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d9b2d-6d68-4342-8a46-002325c5c7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:41.512438 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-178d9b2d-6d68-4342-8a46-002325c5c7f4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:41.514752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-178d9b2d-6d68-4342-8a46-002325c5c7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:41.573262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773fb81-f786-470c-986b-eaba9e39154d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:41.591720 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 385/791] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:15:41.593182 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d9b2d-6d68-4342-8a46-002325c5c7f4 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:41.624341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51df107-d582-4f2c-ab89-03f8e09c868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:41.624977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51df107-d582-4f2c-ab89-03f8e09c868e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:41.625261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51df107-d582-4f2c-ab89-03f8e09c868e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:41.625533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51df107-d582-4f2c-ab89-03f8e09c868e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:41.637762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51df107-d582-4f2c-ab89-03f8e09c868e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:41.639176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51df107-d582-4f2c-ab89-03f8e09c868e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d51df107-d582-4f2c-ab89-03f8e09c868e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:41.669075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:41.670051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:41.671007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:41.671007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:41.671007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:41.671007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:41.671007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:41.671007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:41.671007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:41.671007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:41.671007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:15:41.671007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:41.671007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 407/792] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Sep 12 23:15:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:41.675766 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51df107-d582-4f2c-ab89-03f8e09c868e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:41.696793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:41.704308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d9b2d-6d68-4342-8a46-002325c5c7f4 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:41.741016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81196657-0dfa-4cdc-819f-10af7a0483b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:41.742598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81196657-0dfa-4cdc-819f-10af7a0483b7 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:41.743072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81196657-0dfa-4cdc-819f-10af7a0483b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:41.743536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81196657-0dfa-4cdc-819f-10af7a0483b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:41.744507 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81196657-0dfa-4cdc-819f-10af7a0483b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:41.745032 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81196657-0dfa-4cdc-819f-10af7a0483b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:41.746798 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81196657-0dfa-4cdc-819f-10af7a0483b7 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:41.755595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 386/793] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:41.762124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81196657-0dfa-4cdc-819f-10af7a0483b7 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:41.827971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:41.873351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51df107-d582-4f2c-ab89-03f8e09c868e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:41.909219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81196657-0dfa-4cdc-819f-10af7a0483b7 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:41.954843 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f0809-00f4-46e0-b5c1-19cb02e6fb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:41.955515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f0809-00f4-46e0-b5c1-19cb02e6fb69 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:41.955827 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f0809-00f4-46e0-b5c1-19cb02e6fb69 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:41.956102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f0809-00f4-46e0-b5c1-19cb02e6fb69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:41.968685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a25c70-4d5a-4457-b4a6-3cea5c59cb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:41.969274 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f0809-00f4-46e0-b5c1-19cb02e6fb69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:41.969412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25c70-4d5a-4457-b4a6-3cea5c59cb3a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:41.969591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25c70-4d5a-4457-b4a6-3cea5c59cb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:41.969842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25c70-4d5a-4457-b4a6-3cea5c59cb3a None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:41.970573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a25c70-4d5a-4457-b4a6-3cea5c59cb3a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:41.970744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f0809-00f4-46e0-b5c1-19cb02e6fb69 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:41.970911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a25c70-4d5a-4457-b4a6-3cea5c59cb3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:41.972421 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a25c70-4d5a-4457-b4a6-3cea5c59cb3a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-488f0809-00f4-46e0-b5c1-19cb02e6fb69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:41.986226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:41.987419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:41.988551 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:41.988551 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:41.988551 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:41.990270 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 387/794] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:41.991195 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f0809-00f4-46e0-b5c1-19cb02e6fb69 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:41.991727 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 408/795] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Sep 12 23:15:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:41.997202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a25c70-4d5a-4457-b4a6-3cea5c59cb3a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:42.006321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:42.103630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f0809-00f4-46e0-b5c1-19cb02e6fb69 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:42.115788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a25c70-4d5a-4457-b4a6-3cea5c59cb3a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:42.140255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31076a-43b1-47a0-8082-0fc36506a58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:42.141063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31076a-43b1-47a0-8082-0fc36506a58e None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:42.141432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31076a-43b1-47a0-8082-0fc36506a58e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:42.141801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31076a-43b1-47a0-8082-0fc36506a58e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:42.151833 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f245353-a14d-40c0-b7f5-83faec9f1ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:42.152443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f245353-a14d-40c0-b7f5-83faec9f1ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:42.152718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f245353-a14d-40c0-b7f5-83faec9f1ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:42.152989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f245353-a14d-40c0-b7f5-83faec9f1ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:42.153350 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31076a-43b1-47a0-8082-0fc36506a58e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:42.153617 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f245353-a14d-40c0-b7f5-83faec9f1ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:42.153917 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f245353-a14d-40c0-b7f5-83faec9f1ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:42.155261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f245353-a14d-40c0-b7f5-83faec9f1ea5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:42.156685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-2b31076a-43b1-47a0-8082-0fc36506a58e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:42.160218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:42.162481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:42.163692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:42.163692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:42.163692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:42.163692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:42.163692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:42.163692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:42.163692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:42.163692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:42.163692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:42.163692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:42.163692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:15:42.163692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:42.163692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 388/796] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:15:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:42.163692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31076a-43b1-47a0-8082-0fc36506a58e None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:42.172711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 409/797] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:42.173620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f245353-a14d-40c0-b7f5-83faec9f1ea5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:42.241136 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:42.283679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31076a-43b1-47a0-8082-0fc36506a58e None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:42.288668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f245353-a14d-40c0-b7f5-83faec9f1ea5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:42.320594 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea6af8-5f38-48d1-bc29-518d0cdadbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:42.321370 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea6af8-5f38-48d1-bc29-518d0cdadbdd None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:42.321719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea6af8-5f38-48d1-bc29-518d0cdadbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:42.322057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea6af8-5f38-48d1-bc29-518d0cdadbdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:42.325868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f2981-5c0b-4b33-877a-15f860b4958a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:42.326474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f2981-5c0b-4b33-877a-15f860b4958a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:42.326751 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f2981-5c0b-4b33-877a-15f860b4958a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:42.327021 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f2981-5c0b-4b33-877a-15f860b4958a None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:42.327703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f2981-5c0b-4b33-877a-15f860b4958a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:42.328006 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f2981-5c0b-4b33-877a-15f860b4958a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:42.329359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f2981-5c0b-4b33-877a-15f860b4958a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:42.343397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 410/798] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:15:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:42.344324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f2981-5c0b-4b33-877a-15f860b4958a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:42.393790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea6af8-5f38-48d1-bc29-518d0cdadbdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:42.395226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea6af8-5f38-48d1-bc29-518d0cdadbdd None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:42.403985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:42.449028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f2981-5c0b-4b33-877a-15f860b4958a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:42.468847 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:42.484975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758af3a-e33b-4399-bb5a-821ae4a9dd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:42.485617 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758af3a-e33b-4399-bb5a-821ae4a9dd57 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:42.485901 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758af3a-e33b-4399-bb5a-821ae4a9dd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:42.486175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758af3a-e33b-4399-bb5a-821ae4a9dd57 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:42.495423 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758af3a-e33b-4399-bb5a-821ae4a9dd57 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:42.496837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758af3a-e33b-4399-bb5a-821ae4a9dd57 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:42.515599 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 389/799] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:15:42.515599 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea6af8-5f38-48d1-bc29-518d0cdadbdd None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f758af3a-e33b-4399-bb5a-821ae4a9dd57 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:42.515875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:42.516910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:42.517847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:42.517847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:42.517847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:42.517847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:42.517847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:42.517847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:42.517847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:42.517847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:42.517847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:15:42.517847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:42.517847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 411/800] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Sep 12 23:15:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:42.517847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469622d4-02d9-4f8d-b354-2da1386ecd91 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:42.517847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469622d4-02d9-4f8d-b354-2da1386ecd91 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:42.518551 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469622d4-02d9-4f8d-b354-2da1386ecd91 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:42.518551 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 412/801] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:42.518551 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469622d4-02d9-4f8d-b354-2da1386ecd91 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:42.619113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea6af8-5f38-48d1-bc29-518d0cdadbdd None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:42.632542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469622d4-02d9-4f8d-b354-2da1386ecd91 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:42.649282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:42.656352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f425013-b8b8-4537-824f-dcab326104e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:42.657045 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f425013-b8b8-4537-824f-dcab326104e7 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:42.657261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f425013-b8b8-4537-824f-dcab326104e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:42.657530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f425013-b8b8-4537-824f-dcab326104e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:42.668201 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f425013-b8b8-4537-824f-dcab326104e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:42.669611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f425013-b8b8-4537-824f-dcab326104e7 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:42.671113 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d96e18-10dd-4c87-9e0d-c897d5157985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:42.671649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d96e18-10dd-4c87-9e0d-c897d5157985 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:42.671931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d96e18-10dd-4c87-9e0d-c897d5157985 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:42.672206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d96e18-10dd-4c87-9e0d-c897d5157985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:42.683072 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d96e18-10dd-4c87-9e0d-c897d5157985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:42.684506 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d96e18-10dd-4c87-9e0d-c897d5157985 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-9f425013-b8b8-4537-824f-dcab326104e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:42.684700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:42.685587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:42.686496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:42.686496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:42.686496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:42.686496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 390/802] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:42.687230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b84f1-fb08-4901-b529-77e1db817343 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:42.687230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b84f1-fb08-4901-b529-77e1db817343 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:42.687390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b84f1-fb08-4901-b529-77e1db817343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-77d96e18-10dd-4c87-9e0d-c897d5157985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:42.705304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:42.706336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:42.707276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:42.707276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:42.707276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:42.707276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 413/803] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:42.707276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d96e18-10dd-4c87-9e0d-c897d5157985 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:42.708877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-594b84f1-fb08-4901-b529-77e1db817343 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:42.710154 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-594b84f1-fb08-4901-b529-77e1db817343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:42.730512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:42.786284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 391/804] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:15:42.787772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b84f1-fb08-4901-b529-77e1db817343 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:42.838242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d96e18-10dd-4c87-9e0d-c897d5157985 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:42.874329 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11d339-9468-4089-9e97-777b78d055b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:42.874940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11d339-9468-4089-9e97-777b78d055b1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:42.875222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11d339-9468-4089-9e97-777b78d055b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:42.875496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11d339-9468-4089-9e97-777b78d055b1 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:42.876260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b11d339-9468-4089-9e97-777b78d055b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:42.876566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b11d339-9468-4089-9e97-777b78d055b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:42.879225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b11d339-9468-4089-9e97-777b78d055b1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:42.904756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 414/805] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Sep 12 23:15:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:42.904756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11d339-9468-4089-9e97-777b78d055b1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:42.905059 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b84f1-fb08-4901-b529-77e1db817343 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:42.932030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:42.943788 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454ad20-6184-454e-9112-a87ad023869c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:42.944505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454ad20-6184-454e-9112-a87ad023869c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:42.945662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454ad20-6184-454e-9112-a87ad023869c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:42.946200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454ad20-6184-454e-9112-a87ad023869c None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:42.947258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454ad20-6184-454e-9112-a87ad023869c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:42.947750 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454ad20-6184-454e-9112-a87ad023869c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:42.949135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454ad20-6184-454e-9112-a87ad023869c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:42.961211 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 392/806] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:42.963123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769751ae-a761-41a7-b23c-1890504ac5a2 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:42.963752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769751ae-a761-41a7-b23c-1890504ac5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:42.964286 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769751ae-a761-41a7-b23c-1890504ac5a2 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:42.966413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 393/807] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:42.967296 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769751ae-a761-41a7-b23c-1890504ac5a2 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:42.989133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:43.015592 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11d339-9468-4089-9e97-777b78d055b1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:43.050901 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c22da-edf3-47a9-b0eb-407dc694def6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:43.051434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c22da-edf3-47a9-b0eb-407dc694def6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:43.051756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c22da-edf3-47a9-b0eb-407dc694def6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:43.052034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c22da-edf3-47a9-b0eb-407dc694def6 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:43.052678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c22da-edf3-47a9-b0eb-407dc694def6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:43.052983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c22da-edf3-47a9-b0eb-407dc694def6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:43.054393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c22da-edf3-47a9-b0eb-407dc694def6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:43.065666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 415/808] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:43.066517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c22da-edf3-47a9-b0eb-407dc694def6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:43.087214 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769751ae-a761-41a7-b23c-1890504ac5a2 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:43.120019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b6332-8fa7-40ce-be9b-75b305760216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:43.120637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b6332-8fa7-40ce-be9b-75b305760216 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:43.120913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b6332-8fa7-40ce-be9b-75b305760216 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:43.121184 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b6332-8fa7-40ce-be9b-75b305760216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:43.131101 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b6332-8fa7-40ce-be9b-75b305760216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:43.133149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b6332-8fa7-40ce-be9b-75b305760216 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-887b6332-8fa7-40ce-be9b-75b305760216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:43.146688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:43.147840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:43.148740 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:43.148740 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:43.148740 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:43.148740 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 394/809] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:43.148973 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8599d-aad9-40c4-beef-0d03b61a7270 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:43.149555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8599d-aad9-40c4-beef-0d03b61a7270 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:43.149555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8599d-aad9-40c4-beef-0d03b61a7270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:43.160473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:43.167175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-8cd8599d-aad9-40c4-beef-0d03b61a7270 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:43.168517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8cd8599d-aad9-40c4-beef-0d03b61a7270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:43.176730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c22da-edf3-47a9-b0eb-407dc694def6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:43.210674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f43b5-1cb9-496e-a2dd-9fc6d026832f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:43.211286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f43b5-1cb9-496e-a2dd-9fc6d026832f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:43.211575 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f43b5-1cb9-496e-a2dd-9fc6d026832f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:43.211878 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f43b5-1cb9-496e-a2dd-9fc6d026832f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:43.221372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f43b5-1cb9-496e-a2dd-9fc6d026832f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:43.222722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f43b5-1cb9-496e-a2dd-9fc6d026832f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:43.227190 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-3c1f43b5-1cb9-496e-a2dd-9fc6d026832f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:43.234277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:43.235366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:43.236378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:43.236378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:43.236378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:43.236378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:43.236378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:43.236378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:43.236378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:43.236378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:43.236378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:15:43.236378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:43.236378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 416/810] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:15:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:43.236978 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 395/811] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:15:43.237059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f43b5-1cb9-496e-a2dd-9fc6d026832f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:43.237131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8599d-aad9-40c4-beef-0d03b61a7270 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:43.341345 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8599d-aad9-40c4-beef-0d03b61a7270 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:43.373184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f43b5-1cb9-496e-a2dd-9fc6d026832f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:43.376780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495b11e-5f9f-4fd5-974d-6a3eda5946fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:43.377413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495b11e-5f9f-4fd5-974d-6a3eda5946fc None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:43.377694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495b11e-5f9f-4fd5-974d-6a3eda5946fc None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:43.377965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495b11e-5f9f-4fd5-974d-6a3eda5946fc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:43.378541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495b11e-5f9f-4fd5-974d-6a3eda5946fc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:43.379958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495b11e-5f9f-4fd5-974d-6a3eda5946fc None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-b495b11e-5f9f-4fd5-974d-6a3eda5946fc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:43.389307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:43.390343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:43.391204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 396/812] 10.222.0.27 () {56 vars in 1129 bytes} [Mon Sep 12 23:15:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:43.392480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495b11e-5f9f-4fd5-974d-6a3eda5946fc None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:43.418541 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49762b-4210-4516-907f-67c423a161f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:43.419301 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49762b-4210-4516-907f-67c423a161f9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:43.419783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49762b-4210-4516-907f-67c423a161f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:43.420194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49762b-4210-4516-907f-67c423a161f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:43.427982 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49762b-4210-4516-907f-67c423a161f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:43.431173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49762b-4210-4516-907f-67c423a161f9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6f49762b-4210-4516-907f-67c423a161f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:43.444109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:43.445714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:43.446672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:43.446672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:43.446672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:43.446672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:43.446672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:43.446672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:43.446672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:43.446672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:43.446672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 417/813] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:43.447309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49762b-4210-4516-907f-67c423a161f9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:43.457220 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:43.458923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:43.510915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495b11e-5f9f-4fd5-974d-6a3eda5946fc None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:43.546550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692b070-c475-45d8-8f34-e6f439e91278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:43.547153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692b070-c475-45d8-8f34-e6f439e91278 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:43.547438 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692b070-c475-45d8-8f34-e6f439e91278 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:43.547742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692b070-c475-45d8-8f34-e6f439e91278 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:43.548384 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692b070-c475-45d8-8f34-e6f439e91278 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:43.548688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692b070-c475-45d8-8f34-e6f439e91278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:43.550084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692b070-c475-45d8-8f34-e6f439e91278 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:43.551004 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49762b-4210-4516-907f-67c423a161f9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:43.561491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 397/814] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:43.562364 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692b070-c475-45d8-8f34-e6f439e91278 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:43.589829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae9a8e-1c5d-4996-9c26-d8c6221dfe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:43.590435 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae9a8e-1c5d-4996-9c26-d8c6221dfe12 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:43.590718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae9a8e-1c5d-4996-9c26-d8c6221dfe12 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:43.590994 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae9a8e-1c5d-4996-9c26-d8c6221dfe12 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:43.591800 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae9a8e-1c5d-4996-9c26-d8c6221dfe12 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:43.592061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae9a8e-1c5d-4996-9c26-d8c6221dfe12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:43.593485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae9a8e-1c5d-4996-9c26-d8c6221dfe12 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:43.610125 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 418/815] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Sep 12 23:15:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:43.610980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae9a8e-1c5d-4996-9c26-d8c6221dfe12 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:43.672285 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692b070-c475-45d8-8f34-e6f439e91278 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:43.681588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:43.690399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:43.707509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcfb75-a0b8-4dd5-83b5-d8c195cdedf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:43.708134 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcfb75-a0b8-4dd5-83b5-d8c195cdedf8 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:43.708416 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcfb75-a0b8-4dd5-83b5-d8c195cdedf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:43.708692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcfb75-a0b8-4dd5-83b5-d8c195cdedf8 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:43.709306 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedcfb75-a0b8-4dd5-83b5-d8c195cdedf8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:43.709604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedcfb75-a0b8-4dd5-83b5-d8c195cdedf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:43.710980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedcfb75-a0b8-4dd5-83b5-d8c195cdedf8 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:43.715224 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 398/816] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Sep 12 23:15:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:43.716040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcfb75-a0b8-4dd5-83b5-d8c195cdedf8 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:43.724608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae9a8e-1c5d-4996-9c26-d8c6221dfe12 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:43.758333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff333d-01f6-434f-b43f-5eca5995d281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:43.758933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff333d-01f6-434f-b43f-5eca5995d281 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:43.759205 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff333d-01f6-434f-b43f-5eca5995d281 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:43.759478 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff333d-01f6-434f-b43f-5eca5995d281 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:43.760192 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff333d-01f6-434f-b43f-5eca5995d281 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:43.760498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff333d-01f6-434f-b43f-5eca5995d281 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:43.761904 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff333d-01f6-434f-b43f-5eca5995d281 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:43.773554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 419/817] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:43.774118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff333d-01f6-434f-b43f-5eca5995d281 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:43.815012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcfb75-a0b8-4dd5-83b5-d8c195cdedf8 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:43.847950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e7869-bdea-4c8a-84a4-6cff723d2258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:43.848504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e7869-bdea-4c8a-84a4-6cff723d2258 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:43.848781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e7869-bdea-4c8a-84a4-6cff723d2258 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:43.849051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e7869-bdea-4c8a-84a4-6cff723d2258 None admin] PATH_INFO: `/v3/services/network` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:43.849621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e7869-bdea-4c8a-84a4-6cff723d2258 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:43.850994 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e7869-bdea-4c8a-84a4-6cff723d2258 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-e29e7869-bdea-4c8a-84a4-6cff723d2258 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:43.859470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:43.863801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:43.864962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 399/818] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:43 2022] GET /identity/v3/services/network => generated 88 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:43.866163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e7869-bdea-4c8a-84a4-6cff723d2258 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:43.871815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff333d-01f6-434f-b43f-5eca5995d281 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:43.894228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:43.905554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6380d-3552-457d-b070-94aed4ae0eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:43.906169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6380d-3552-457d-b070-94aed4ae0eda None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:43.906445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6380d-3552-457d-b070-94aed4ae0eda None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:43.906714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6380d-3552-457d-b070-94aed4ae0eda None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:43.909723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:43.914524 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6380d-3552-457d-b070-94aed4ae0eda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:43.916000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6380d-3552-457d-b070-94aed4ae0eda None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-0bb6380d-3552-457d-b070-94aed4ae0eda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:43.927843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:43.929943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:43.931289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:43.931289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:43.931289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:43.931289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:43.931289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:43.931289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:43.931289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:43.931289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:43.931289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 420/819] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:43.931289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6380d-3552-457d-b070-94aed4ae0eda None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:43.969564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e7869-bdea-4c8a-84a4-6cff723d2258 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:44.002310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2580f6-9345-4150-9117-45892092c068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:44.003014 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2580f6-9345-4150-9117-45892092c068 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:44.003296 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2580f6-9345-4150-9117-45892092c068 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:44.003600 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2580f6-9345-4150-9117-45892092c068 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:44.014555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2580f6-9345-4150-9117-45892092c068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:44.016131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2580f6-9345-4150-9117-45892092c068 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-3b2580f6-9345-4150-9117-45892092c068 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:44.029752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:44.030839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:44.031771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:44.031771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:44.031771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:44.031960 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6380d-3552-457d-b070-94aed4ae0eda None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:44.032041 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 400/820] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:44.032041 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2580f6-9345-4150-9117-45892092c068 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:44.064135 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bc9ea-6716-467a-bf2e-d5b3a1c925c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:44.064742 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bc9ea-6716-467a-bf2e-d5b3a1c925c6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:44.065024 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bc9ea-6716-467a-bf2e-d5b3a1c925c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:44.065294 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bc9ea-6716-467a-bf2e-d5b3a1c925c6 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:44.065922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bc9ea-6716-467a-bf2e-d5b3a1c925c6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:44.066222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bc9ea-6716-467a-bf2e-d5b3a1c925c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:44.067595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bc9ea-6716-467a-bf2e-d5b3a1c925c6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:44.072805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 421/821] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Sep 12 23:15:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:44.073567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bc9ea-6716-467a-bf2e-d5b3a1c925c6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:44.120822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:44.129625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:44.135157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2580f6-9345-4150-9117-45892092c068 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:44.168316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4481d-9568-4a81-b73a-d2ed3e4b1ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:44.168915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4481d-9568-4a81-b73a-d2ed3e4b1ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:44.169191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4481d-9568-4a81-b73a-d2ed3e4b1ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:44.169461 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4481d-9568-4a81-b73a-d2ed3e4b1ae0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:44.170175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4481d-9568-4a81-b73a-d2ed3e4b1ae0 None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=f5fa28ffdafd435cb2d93bd453add3cb) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:44.170482 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4481d-9568-4a81-b73a-d2ed3e4b1ae0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:44.171921 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4481d-9568-4a81-b73a-d2ed3e4b1ae0 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:44.172663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bc9ea-6716-467a-bf2e-d5b3a1c925c6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:44.177413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 401/822] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:44 2022] GET /identity/v3/role_assignments?user.id=f5fa28ffdafd435cb2d93bd453add3cb&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=23538dc4c770454e80dd11ed8e498539 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:44.178175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4481d-9568-4a81-b73a-d2ed3e4b1ae0 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:44.209822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4d65d-79ff-4de8-8644-f3662d889e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:44.211025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4d65d-79ff-4de8-8644-f3662d889e3c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:44.211311 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4d65d-79ff-4de8-8644-f3662d889e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:44.211606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4d65d-79ff-4de8-8644-f3662d889e3c None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:44.212232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4d65d-79ff-4de8-8644-f3662d889e3c None admin] RBAC: Adding query filter params (name=network) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:44.212536 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4d65d-79ff-4de8-8644-f3662d889e3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:44.213917 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4d65d-79ff-4de8-8644-f3662d889e3c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:44.223622 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 422/823] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Sep 12 23:15:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:44.223622 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4d65d-79ff-4de8-8644-f3662d889e3c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:44.305427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4481d-9568-4a81-b73a-d2ed3e4b1ae0 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:44.334969 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4d65d-79ff-4de8-8644-f3662d889e3c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:44.344069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2801f-3003-4759-959a-d1467e324b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:44.344698 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2801f-3003-4759-959a-d1467e324b1f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:44.345101 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2801f-3003-4759-959a-d1467e324b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:44.345379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2801f-3003-4759-959a-d1467e324b1f None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:44.346085 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2801f-3003-4759-959a-d1467e324b1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:44.346392 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2801f-3003-4759-959a-d1467e324b1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:44.348007 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2801f-3003-4759-959a-d1467e324b1f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:44.349558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:44.362823 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 402/824] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:44.363544 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2801f-3003-4759-959a-d1467e324b1f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:44.379746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca4d18-e359-4a89-a9a7-f089cb2fdbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:44.380295 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca4d18-e359-4a89-a9a7-f089cb2fdbcb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:44.380580 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca4d18-e359-4a89-a9a7-f089cb2fdbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:44.380857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca4d18-e359-4a89-a9a7-f089cb2fdbcb None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:44.381506 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca4d18-e359-4a89-a9a7-f089cb2fdbcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:44.381812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca4d18-e359-4a89-a9a7-f089cb2fdbcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:44.383753 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca4d18-e359-4a89-a9a7-f089cb2fdbcb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:44.390945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:44.397107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 423/825] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:44.398000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca4d18-e359-4a89-a9a7-f089cb2fdbcb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:44.474810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2801f-3003-4759-959a-d1467e324b1f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:44.511428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca4d18-e359-4a89-a9a7-f089cb2fdbcb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:44.511966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271655e-0ee0-4b01-a520-ebe81370fdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:44.512572 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271655e-0ee0-4b01-a520-ebe81370fdb1 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:44.512858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271655e-0ee0-4b01-a520-ebe81370fdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:44.513129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271655e-0ee0-4b01-a520-ebe81370fdb1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:44.513784 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271655e-0ee0-4b01-a520-ebe81370fdb1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:44.515147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271655e-0ee0-4b01-a520-ebe81370fdb1 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:44.537072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 403/826] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:15:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:15:44.538515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271655e-0ee0-4b01-a520-ebe81370fdb1 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:44.566624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13501f-bf96-4567-a1e4-aaefdffcec11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:44.567713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13501f-bf96-4567-a1e4-aaefdffcec11 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:44.568411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13501f-bf96-4567-a1e4-aaefdffcec11 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:44.568835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13501f-bf96-4567-a1e4-aaefdffcec11 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:44.570247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13501f-bf96-4567-a1e4-aaefdffcec11 None admin] RBAC: Adding query filter params (type=network) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:44.570890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13501f-bf96-4567-a1e4-aaefdffcec11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:44.572690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13501f-bf96-4567-a1e4-aaefdffcec11 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:44.579018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 424/827] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Sep 12 23:15:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:44.580473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13501f-bf96-4567-a1e4-aaefdffcec11 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:44.617159 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:44.650426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:44.681709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271655e-0ee0-4b01-a520-ebe81370fdb1 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:44.713794 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13501f-bf96-4567-a1e4-aaefdffcec11 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:44.731765 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0cc29-3297-4d59-847e-cf16fa93aa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:44.732746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0cc29-3297-4d59-847e-cf16fa93aa28 None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:44.733216 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0cc29-3297-4d59-847e-cf16fa93aa28 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:44.733643 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0cc29-3297-4d59-847e-cf16fa93aa28 None admin] PATH_INFO: `/v3/projects/32bbb936501846e8914acf07890c2891/users/0212eb5841a04075a0fd7ba853231267/roles/5c7e1f557ae04250a01db783f3c27a5f` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:44.752875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41bf9c-e254-43e7-8b92-bd61a1425f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:44.753088 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0cc29-3297-4d59-847e-cf16fa93aa28 None admin] RBAC: Authorizing `identity:create_grant(project_id=32bbb936501846e8914acf07890c2891, user_id=0212eb5841a04075a0fd7ba853231267, role_id=5c7e1f557ae04250a01db783f3c27a5f)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:44.753517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41bf9c-e254-43e7-8b92-bd61a1425f03 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:44.753795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41bf9c-e254-43e7-8b92-bd61a1425f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:44.754066 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41bf9c-e254-43e7-8b92-bd61a1425f03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:44.754388 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0cc29-3297-4d59-847e-cf16fa93aa28 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:44.761657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41bf9c-e254-43e7-8b92-bd61a1425f03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:44.763704 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41bf9c-e254-43e7-8b92-bd61a1425f03 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:44.770110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 404/828] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:15:44 2022] PUT /identity/v3/projects/32bbb936501846e8914acf07890c2891/users/0212eb5841a04075a0fd7ba853231267/roles/5c7e1f557ae04250a01db783f3c27a5f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:15:44.770938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0cc29-3297-4d59-847e-cf16fa93aa28 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-af41bf9c-e254-43e7-8b92-bd61a1425f03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:44.776822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:44.777765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:44.778746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:44.778746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:44.778746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:44.778746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:44.778746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:44.778746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:44.778746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:44.778746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:44.778746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 425/829] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:44.785802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41bf9c-e254-43e7-8b92-bd61a1425f03 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:44.903891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:44.916103 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41bf9c-e254-43e7-8b92-bd61a1425f03 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:44.922284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0cc29-3297-4d59-847e-cf16fa93aa28 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:44.969695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cbadf-56a1-4414-9ec8-58a231298bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:44.969920 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cbadf-56a1-4414-9ec8-58a231298bad None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:44.972194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cbadf-56a1-4414-9ec8-58a231298bad None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:44.972194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819cbadf-56a1-4414-9ec8-58a231298bad None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:44.973709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819cbadf-56a1-4414-9ec8-58a231298bad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:44.974230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:44.975041 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819cbadf-56a1-4414-9ec8-58a231298bad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:44.978668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819cbadf-56a1-4414-9ec8-58a231298bad None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:44.999559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d511fa8-3c56-4200-a515-e0a22efbdbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:45.001829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d511fa8-3c56-4200-a515-e0a22efbdbe1 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:45.001829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d511fa8-3c56-4200-a515-e0a22efbdbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:45.001829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d511fa8-3c56-4200-a515-e0a22efbdbe1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:45.002732 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d511fa8-3c56-4200-a515-e0a22efbdbe1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:45.004287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d511fa8-3c56-4200-a515-e0a22efbdbe1 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:45.017701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 426/830] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:45.023503 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cbadf-56a1-4414-9ec8-58a231298bad None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:45.039674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 405/831] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:15:44 2022] POST /identity/v3/endpoints => generated 337 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:15:45.147791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819cbadf-56a1-4414-9ec8-58a231298bad None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:45.209172 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00eaf16-22ca-449f-9c87-b1b1517f22d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:45.209854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00eaf16-22ca-449f-9c87-b1b1517f22d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:45.210672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00eaf16-22ca-449f-9c87-b1b1517f22d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:45.210949 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00eaf16-22ca-449f-9c87-b1b1517f22d4 None admin] PATH_INFO: `/v3/projects/32bbb936501846e8914acf07890c2891/users/c1f6bb09fd7f4905a1693ef7cb6f088e/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:45.213003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00eaf16-22ca-449f-9c87-b1b1517f22d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=32bbb936501846e8914acf07890c2891, user_id=c1f6bb09fd7f4905a1693ef7cb6f088e, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:45.214583 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00eaf16-22ca-449f-9c87-b1b1517f22d4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:45.234689 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 427/832] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:15:45 2022] PUT /identity/v3/projects/32bbb936501846e8914acf07890c2891/users/c1f6bb09fd7f4905a1693ef7cb6f088e/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:15:46.496999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547665b-1a7c-4b60-a6fe-71a8aaf42584 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:46.497541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547665b-1a7c-4b60-a6fe-71a8aaf42584 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:46.497541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547665b-1a7c-4b60-a6fe-71a8aaf42584 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:46.499129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 406/833] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:46.504422 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b15e-5059-4663-9199-0392e72a6841 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:46.504810 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b15e-5059-4663-9199-0392e72a6841 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:46.504980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b15e-5059-4663-9199-0392e72a6841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:46.523849 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a616b15e-5059-4663-9199-0392e72a6841 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:46.525038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a616b15e-5059-4663-9199-0392e72a6841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:46.527088 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:46.603692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 428/834] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:15:46.862704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547665b-1a7c-4b60-a6fe-71a8aaf42584 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:46.980960 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547665b-1a7c-4b60-a6fe-71a8aaf42584 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:47.017086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a73785-b5ec-4532-96e3-c5d9971db369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:47.017717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a73785-b5ec-4532-96e3-c5d9971db369 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:47.017991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a73785-b5ec-4532-96e3-c5d9971db369 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:47.018258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a73785-b5ec-4532-96e3-c5d9971db369 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:47.018839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a73785-b5ec-4532-96e3-c5d9971db369 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:47.020268 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a73785-b5ec-4532-96e3-c5d9971db369 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-24a73785-b5ec-4532-96e3-c5d9971db369 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:47.029796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:47.031444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:47.032386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 407/835] 10.222.0.27 () {56 vars in 1132 bytes} [Mon Sep 12 23:15:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:47.039936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616b15e-5059-4663-9199-0392e72a6841 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:47.075952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b87d58-06e1-48f9-9d96-ad67b12c4a75 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:47.076297 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b87d58-06e1-48f9-9d96-ad67b12c4a75 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:47.076511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b87d58-06e1-48f9-9d96-ad67b12c4a75 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:47.078206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 408/836] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:47.084019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265ec4-83fa-40b3-9e74-6137e6d6068a None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:47.084334 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265ec4-83fa-40b3-9e74-6137e6d6068a None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:47.084559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265ec4-83fa-40b3-9e74-6137e6d6068a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:47.105073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-75265ec4-83fa-40b3-9e74-6137e6d6068a None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:47.106266 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-75265ec4-83fa-40b3-9e74-6137e6d6068a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:47.154304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616b15e-5059-4663-9199-0392e72a6841 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:47.193595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd6b8e-847d-41fe-a998-7ab051cf8cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:47.194881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd6b8e-847d-41fe-a998-7ab051cf8cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:47.195061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd6b8e-847d-41fe-a998-7ab051cf8cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:47.195297 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd6b8e-847d-41fe-a998-7ab051cf8cf3 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:47.195979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd6b8e-847d-41fe-a998-7ab051cf8cf3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:47.196287 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd6b8e-847d-41fe-a998-7ab051cf8cf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:47.197694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd6b8e-847d-41fe-a998-7ab051cf8cf3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:47.203227 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 429/837] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Sep 12 23:15:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:47.204288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd1e29-629d-4804-8682-505c7856c7d1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:47.204567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd1e29-629d-4804-8682-505c7856c7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:47.204986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd1e29-629d-4804-8682-505c7856c7d1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:47.206335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 430/838] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:47.211056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd1e29-629d-4804-8682-505c7856c7d1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:47.214275 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:47.266895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:47.303951 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 409/839] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:15:47.305164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61361681-806f-4f8f-8704-d23e9f45685c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:47.305596 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61361681-806f-4f8f-8704-d23e9f45685c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:47.305731 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61361681-806f-4f8f-8704-d23e9f45685c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:47.328263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd1e29-629d-4804-8682-505c7856c7d1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:47.331962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-61361681-806f-4f8f-8704-d23e9f45685c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:47.333226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-61361681-806f-4f8f-8704-d23e9f45685c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:47.368952 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a4d7a-a2ab-4cab-96aa-26e649898458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:47.369558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a4d7a-a2ab-4cab-96aa-26e649898458 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:47.369832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a4d7a-a2ab-4cab-96aa-26e649898458 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:47.370101 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a4d7a-a2ab-4cab-96aa-26e649898458 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:47.370725 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a4d7a-a2ab-4cab-96aa-26e649898458 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:47.371024 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a4d7a-a2ab-4cab-96aa-26e649898458 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:47.372422 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a4d7a-a2ab-4cab-96aa-26e649898458 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:47.380511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 431/840] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Sep 12 23:15:47 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:47.420975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 410/841] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:15:47.579552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a4d7a-a2ab-4cab-96aa-26e649898458 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:47.586307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486c7ea-f2ad-41af-bbe6-9bf4118a87c2 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:47.586789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486c7ea-f2ad-41af-bbe6-9bf4118a87c2 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:47.586879 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486c7ea-f2ad-41af-bbe6-9bf4118a87c2 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:47.588482 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 411/842] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:15:47.596777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b3459-7b41-4077-a304-fbfd10e92ca2 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:47.597186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b3459-7b41-4077-a304-fbfd10e92ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:47.597557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b3459-7b41-4077-a304-fbfd10e92ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:47.615914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-2c5b3459-7b41-4077-a304-fbfd10e92ca2 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:47.617762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2c5b3459-7b41-4077-a304-fbfd10e92ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:47.656178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:47.690370 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 412/843] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:15:47.694558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b3459-7b41-4077-a304-fbfd10e92ca2 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:47.698576 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a4d7a-a2ab-4cab-96aa-26e649898458 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:47.736344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:47.742545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9142b-e127-4b8b-bc1e-cdcdced581b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:47.743179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9142b-e127-4b8b-bc1e-cdcdced581b0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:47.743456 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9142b-e127-4b8b-bc1e-cdcdced581b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:47.743864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9142b-e127-4b8b-bc1e-cdcdced581b0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:47.744451 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9142b-e127-4b8b-bc1e-cdcdced581b0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:47.745922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9142b-e127-4b8b-bc1e-cdcdced581b0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-60f9142b-e127-4b8b-bc1e-cdcdced581b0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:47.756567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:47.758428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:47.759820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 432/844] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:15:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:47.767600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9142b-e127-4b8b-bc1e-cdcdced581b0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:47.823628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b3459-7b41-4077-a304-fbfd10e92ca2 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:47.868882 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03bc95-4cdc-4465-853b-42e09ea13cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:47.870032 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03bc95-4cdc-4465-853b-42e09ea13cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:47.870311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03bc95-4cdc-4465-853b-42e09ea13cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:47.870581 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03bc95-4cdc-4465-853b-42e09ea13cb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:47.882888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9142b-e127-4b8b-bc1e-cdcdced581b0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:47.885001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03bc95-4cdc-4465-853b-42e09ea13cb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-0e03bc95-4cdc-4465-853b-42e09ea13cb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:47.891242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:47.892954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:47.894115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:47.894115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:47.894115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:47.894115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:47.894115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:47.894115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:47.894115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:47.894115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:47.894115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:47.894115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:47.894115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:15:47.894115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:47.894115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 413/845] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:15:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:15:47.901956 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03bc95-4cdc-4465-853b-42e09ea13cb4 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:47.926996 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:47.934034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a17328-f4b3-4ff7-80a1-ccb97ffc0479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:47.934468 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a17328-f4b3-4ff7-80a1-ccb97ffc0479 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:47.934749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a17328-f4b3-4ff7-80a1-ccb97ffc0479 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:47.935022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a17328-f4b3-4ff7-80a1-ccb97ffc0479 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:47.935690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a17328-f4b3-4ff7-80a1-ccb97ffc0479 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:47.935993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a17328-f4b3-4ff7-80a1-ccb97ffc0479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:47.937347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a17328-f4b3-4ff7-80a1-ccb97ffc0479 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:47.947263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 433/846] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:15:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:47.949267 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a17328-f4b3-4ff7-80a1-ccb97ffc0479 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:48.023731 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:48.028596 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03bc95-4cdc-4465-853b-42e09ea13cb4 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:48.063711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a17328-f4b3-4ff7-80a1-ccb97ffc0479 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:48.064287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955237e-0edd-4503-a42c-37b903327b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:48.064897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955237e-0edd-4503-a42c-37b903327b97 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:48.065181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955237e-0edd-4503-a42c-37b903327b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:48.065453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955237e-0edd-4503-a42c-37b903327b97 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:48.066116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955237e-0edd-4503-a42c-37b903327b97 None admin] RBAC: Adding query filter params (name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:48.066419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955237e-0edd-4503-a42c-37b903327b97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:48.067821 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955237e-0edd-4503-a42c-37b903327b97 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:48.081264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 414/847] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:15:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:48.081264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955237e-0edd-4503-a42c-37b903327b97 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:48.104109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63a3cd-a086-4384-a143-c9ed483e75f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:48.104924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63a3cd-a086-4384-a143-c9ed483e75f7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:48.105272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63a3cd-a086-4384-a143-c9ed483e75f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:48.105638 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63a3cd-a086-4384-a143-c9ed483e75f7 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:48.106340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63a3cd-a086-4384-a143-c9ed483e75f7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:48.106717 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63a3cd-a086-4384-a143-c9ed483e75f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:48.108182 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63a3cd-a086-4384-a143-c9ed483e75f7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:48.112790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 434/848] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:15:47 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:48.120678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63a3cd-a086-4384-a143-c9ed483e75f7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:48.192323 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955237e-0edd-4503-a42c-37b903327b97 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:48.211598 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:48.235882 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d7620-1bd5-4d37-895c-58fcbc72ada8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:48.236924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d7620-1bd5-4d37-895c-58fcbc72ada8 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:48.237381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d7620-1bd5-4d37-895c-58fcbc72ada8 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:48.237855 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d7620-1bd5-4d37-895c-58fcbc72ada8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:48.252162 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d7620-1bd5-4d37-895c-58fcbc72ada8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-444d7620-1bd5-4d37-895c-58fcbc72ada8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:48.259390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:48.260440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:48.261284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:48.261284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:48.261284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:48.261284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:48.261284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:48.261284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:48.261284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:48.261284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:48.261284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:48.261284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:48.261284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:15:48.261284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:48.268593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63a3cd-a086-4384-a143-c9ed483e75f7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:48.269352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 415/849] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:15:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:48.270487 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d7620-1bd5-4d37-895c-58fcbc72ada8 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:48.298511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:48.310176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651f1a1-162c-416f-a0d0-010b886f162e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:48.310813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651f1a1-162c-416f-a0d0-010b886f162e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:48.311245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651f1a1-162c-416f-a0d0-010b886f162e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:48.311521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651f1a1-162c-416f-a0d0-010b886f162e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:48.324767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651f1a1-162c-416f-a0d0-010b886f162e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:48.326199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651f1a1-162c-416f-a0d0-010b886f162e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5651f1a1-162c-416f-a0d0-010b886f162e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:48.343395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:48.344466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:48.345368 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:48.345368 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:48.345368 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:48.345368 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 435/850] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:48.351770 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651f1a1-162c-416f-a0d0-010b886f162e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:48.393534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d7620-1bd5-4d37-895c-58fcbc72ada8 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:48.427275 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b71a7-60b9-495d-8b13-733ca43cb8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:48.427903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b71a7-60b9-495d-8b13-733ca43cb8aa None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:48.428175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b71a7-60b9-495d-8b13-733ca43cb8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:48.428503 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b71a7-60b9-495d-8b13-733ca43cb8aa None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:48.429713 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b71a7-60b9-495d-8b13-733ca43cb8aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:48.430013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b71a7-60b9-495d-8b13-733ca43cb8aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:48.431432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b71a7-60b9-495d-8b13-733ca43cb8aa None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:48.444539 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 416/851] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:15:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:48.445537 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b71a7-60b9-495d-8b13-733ca43cb8aa None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:48.460924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651f1a1-162c-416f-a0d0-010b886f162e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:48.476422 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:48.504732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a276b0-2f26-442d-ab5f-f2a79552231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:48.505336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a276b0-2f26-442d-ab5f-f2a79552231a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:48.505871 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a276b0-2f26-442d-ab5f-f2a79552231a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:48.506146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a276b0-2f26-442d-ab5f-f2a79552231a None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:48.507257 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a276b0-2f26-442d-ab5f-f2a79552231a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:48.507842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a276b0-2f26-442d-ab5f-f2a79552231a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:48.509315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a276b0-2f26-442d-ab5f-f2a79552231a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:48.523779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 436/852] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:48.529069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a276b0-2f26-442d-ab5f-f2a79552231a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:48.535688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:48.559069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b71a7-60b9-495d-8b13-733ca43cb8aa None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:48.594428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a0da2-64d4-44a6-915b-a9aca85175e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:48.595111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a0da2-64d4-44a6-915b-a9aca85175e3 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:48.595407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a0da2-64d4-44a6-915b-a9aca85175e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:48.595759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a0da2-64d4-44a6-915b-a9aca85175e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:48.607136 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a0da2-64d4-44a6-915b-a9aca85175e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:48.608886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a0da2-64d4-44a6-915b-a9aca85175e3 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-fd4a0da2-64d4-44a6-915b-a9aca85175e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:48.624512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:48.625615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:48.626497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:48.626497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:48.626497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:48.626497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 417/853] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:15:48.633010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a0da2-64d4-44a6-915b-a9aca85175e3 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:48.654543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a276b0-2f26-442d-ab5f-f2a79552231a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:48.700334 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455d107-7924-4a44-9dc2-08a32ef8df6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:48.701322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455d107-7924-4a44-9dc2-08a32ef8df6b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:48.701819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455d107-7924-4a44-9dc2-08a32ef8df6b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:48.702347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455d107-7924-4a44-9dc2-08a32ef8df6b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:48.711480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455d107-7924-4a44-9dc2-08a32ef8df6b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:48.713430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455d107-7924-4a44-9dc2-08a32ef8df6b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6455d107-7924-4a44-9dc2-08a32ef8df6b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:48.727676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:48.728637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:48.729578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:48.729578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:48.729578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:48.729578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:48.729578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:48.729578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:48.729578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:48.729578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:48.729578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:15:48.729578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:48.737519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 437/854] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Sep 12 23:15:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:15:48.738563 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455d107-7924-4a44-9dc2-08a32ef8df6b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:48.747113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a0da2-64d4-44a6-915b-a9aca85175e3 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:48.773603 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:48.789553 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f296b7-181e-44a2-8200-0270372db2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:48.790160 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f296b7-181e-44a2-8200-0270372db2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:48.790438 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f296b7-181e-44a2-8200-0270372db2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:48.790709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f296b7-181e-44a2-8200-0270372db2c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:48.791730 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f296b7-181e-44a2-8200-0270372db2c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:48.792043 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f296b7-181e-44a2-8200-0270372db2c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:48.793774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f296b7-181e-44a2-8200-0270372db2c5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:48.795450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:48.813880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 418/855] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:48.816052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f296b7-181e-44a2-8200-0270372db2c5 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:48.863989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455d107-7924-4a44-9dc2-08a32ef8df6b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:48.905445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34806e5-ff79-4772-a5fb-c9d0e9bec4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:48.906429 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34806e5-ff79-4772-a5fb-c9d0e9bec4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:48.907025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34806e5-ff79-4772-a5fb-c9d0e9bec4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:48.907752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34806e5-ff79-4772-a5fb-c9d0e9bec4b1 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:48.908671 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34806e5-ff79-4772-a5fb-c9d0e9bec4b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:48.909460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34806e5-ff79-4772-a5fb-c9d0e9bec4b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:48.911666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34806e5-ff79-4772-a5fb-c9d0e9bec4b1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:48.940466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 438/856] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Sep 12 23:15:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:48.942026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34806e5-ff79-4772-a5fb-c9d0e9bec4b1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:48.944980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f296b7-181e-44a2-8200-0270372db2c5 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:48.990599 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77069d4b-0583-48bb-b3d7-26c1e1918c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:48.991799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77069d4b-0583-48bb-b3d7-26c1e1918c22 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:48.992094 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77069d4b-0583-48bb-b3d7-26c1e1918c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:48.992654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77069d4b-0583-48bb-b3d7-26c1e1918c22 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:48.999707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77069d4b-0583-48bb-b3d7-26c1e1918c22 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:49.001155 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77069d4b-0583-48bb-b3d7-26c1e1918c22 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-77069d4b-0583-48bb-b3d7-26c1e1918c22 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:49.013052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:49.013999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:49.014913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:49.014913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:49.014913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:49.014913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:49.014913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:49.014913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:49.014913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:49.014913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:49.014913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:15:49.014913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:49.018918 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 419/857] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:15:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:49.019785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77069d4b-0583-48bb-b3d7-26c1e1918c22 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:49.052933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:49.070999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34806e5-ff79-4772-a5fb-c9d0e9bec4b1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:49.083459 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:49.117306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8737a96-3db8-473e-b20d-939c779e22a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:49.118778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8737a96-3db8-473e-b20d-939c779e22a8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:49.119412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8737a96-3db8-473e-b20d-939c779e22a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:49.120010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8737a96-3db8-473e-b20d-939c779e22a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:49.132708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77069d4b-0583-48bb-b3d7-26c1e1918c22 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:49.179226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eea174-0723-46d3-85ea-07205cc304dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:49.180523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eea174-0723-46d3-85ea-07205cc304dc None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:49.181022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eea174-0723-46d3-85ea-07205cc304dc None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:49.181738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eea174-0723-46d3-85ea-07205cc304dc None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:49.183308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eea174-0723-46d3-85ea-07205cc304dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:49.183846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eea174-0723-46d3-85ea-07205cc304dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:49.186425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eea174-0723-46d3-85ea-07205cc304dc None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:49.198177 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8737a96-3db8-473e-b20d-939c779e22a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:49.200550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8737a96-3db8-473e-b20d-939c779e22a8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:49.210905 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 420/858] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Sep 12 23:15:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:49.216737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eea174-0723-46d3-85ea-07205cc304dc None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:49.310693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:49.331718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 439/859] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:48 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 23:15:49.332924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8737a96-3db8-473e-b20d-939c779e22a8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:49.351247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eea174-0723-46d3-85ea-07205cc304dc None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:49.368063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:49.396398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1be9c3-8ed9-4c9c-a4a8-7cda15fca848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:49.397633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1be9c3-8ed9-4c9c-a4a8-7cda15fca848 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:49.398417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1be9c3-8ed9-4c9c-a4a8-7cda15fca848 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:49.399196 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1be9c3-8ed9-4c9c-a4a8-7cda15fca848 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:49.446911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8737a96-3db8-473e-b20d-939c779e22a8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:49.480212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1be9c3-8ed9-4c9c-a4a8-7cda15fca848 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:49.481722 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1be9c3-8ed9-4c9c-a4a8-7cda15fca848 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:49.493433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0adee5d-6659-44ba-8ae0-aa9179a7c2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:49.494885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adee5d-6659-44ba-8ae0-aa9179a7c2ff None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:49.495390 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adee5d-6659-44ba-8ae0-aa9179a7c2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:49.496083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adee5d-6659-44ba-8ae0-aa9179a7c2ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:49.509404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0adee5d-6659-44ba-8ae0-aa9179a7c2ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:49.511409 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0adee5d-6659-44ba-8ae0-aa9179a7c2ff None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c0adee5d-6659-44ba-8ae0-aa9179a7c2ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:49.530459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:49.531746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:49.533174 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:49.533174 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:49.533174 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:49.534846 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 440/860] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:49.541285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0adee5d-6659-44ba-8ae0-aa9179a7c2ff None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:49.592312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:49.598344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 421/861] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:49 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:15:49.599491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4596d9b-118f-472d-aa5e-ff985e8a8398 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:49.599802 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4596d9b-118f-472d-aa5e-ff985e8a8398 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:49.600085 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4596d9b-118f-472d-aa5e-ff985e8a8398 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:49.605827 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 422/862] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:49.605827 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4596d9b-118f-472d-aa5e-ff985e8a8398 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:49.617466 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:49.654126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0adee5d-6659-44ba-8ae0-aa9179a7c2ff None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:49.691676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f3f35-4115-4960-a15b-b0ad2591019f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:49.692314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f3f35-4115-4960-a15b-b0ad2591019f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:49.692544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f3f35-4115-4960-a15b-b0ad2591019f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:49.692806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f3f35-4115-4960-a15b-b0ad2591019f None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:49.693436 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f3f35-4115-4960-a15b-b0ad2591019f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:49.693735 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f3f35-4115-4960-a15b-b0ad2591019f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:49.695178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f3f35-4115-4960-a15b-b0ad2591019f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:49.707605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 441/863] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:49.708194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df42fb-403b-45e6-b01d-f559782802b4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:49.708476 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df42fb-403b-45e6-b01d-f559782802b4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:49.708745 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df42fb-403b-45e6-b01d-f559782802b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:49.715480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4596d9b-118f-472d-aa5e-ff985e8a8398 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:49.726959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a2df42fb-403b-45e6-b01d-f559782802b4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:49.728159 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a2df42fb-403b-45e6-b01d-f559782802b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:49.752331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67950ba3-a4be-4912-9605-51d7f8fe735d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:49.752956 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67950ba3-a4be-4912-9605-51d7f8fe735d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:49.753281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67950ba3-a4be-4912-9605-51d7f8fe735d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:49.753552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67950ba3-a4be-4912-9605-51d7f8fe735d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:49.764304 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67950ba3-a4be-4912-9605-51d7f8fe735d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:49.765738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67950ba3-a4be-4912-9605-51d7f8fe735d None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-67950ba3-a4be-4912-9605-51d7f8fe735d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:49.781524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:49.782736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:49.783673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:49.783673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:49.783673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:49.783673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 423/864] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:49.783673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67950ba3-a4be-4912-9605-51d7f8fe735d None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:49.807607 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 442/865] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:15:49.807607 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df42fb-403b-45e6-b01d-f559782802b4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:49.838967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:49.870757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:49.915521 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67950ba3-a4be-4912-9605-51d7f8fe735d None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:49.929337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df42fb-403b-45e6-b01d-f559782802b4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:49.954962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195a92a-f0db-4c8b-872b-159d15ab44aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:49.955969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195a92a-f0db-4c8b-872b-159d15ab44aa None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:49.956454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195a92a-f0db-4c8b-872b-159d15ab44aa None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:49.956878 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195a92a-f0db-4c8b-872b-159d15ab44aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:49.964872 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195a92a-f0db-4c8b-872b-159d15ab44aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:49.966663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195a92a-f0db-4c8b-872b-159d15ab44aa None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:49.971853 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f9fe6-6619-4445-a5b9-b8c154873068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:49.972462 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f9fe6-6619-4445-a5b9-b8c154873068 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:49.972738 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f9fe6-6619-4445-a5b9-b8c154873068 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:49.973009 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f9fe6-6619-4445-a5b9-b8c154873068 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:49.973651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f9fe6-6619-4445-a5b9-b8c154873068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:49.973948 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f9fe6-6619-4445-a5b9-b8c154873068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:49.975693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f9fe6-6619-4445-a5b9-b8c154873068 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-4195a92a-f0db-4c8b-872b-159d15ab44aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:49.980494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:49.981559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:49.982514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:49.982514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:49.982514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:49.982514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:49.982514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:49.982514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:49.982514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:49.982514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:49.987971 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 424/866] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:49.990324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195a92a-f0db-4c8b-872b-159d15ab44aa None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:49.992602 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 443/867] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:49.993480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f9fe6-6619-4445-a5b9-b8c154873068 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:50.107332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:50.119981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f9fe6-6619-4445-a5b9-b8c154873068 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:50.123467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195a92a-f0db-4c8b-872b-159d15ab44aa None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:50.149549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:50.157895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d2614-8f6b-415b-9d28-3261f6893f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:50.158519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d2614-8f6b-415b-9d28-3261f6893f43 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:50.158799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d2614-8f6b-415b-9d28-3261f6893f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:50.159129 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d2614-8f6b-415b-9d28-3261f6893f43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:50.162603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4d674-1d7f-4f1d-842b-e2420944e0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:50.163368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4d674-1d7f-4f1d-842b-e2420944e0bb None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:50.163772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4d674-1d7f-4f1d-842b-e2420944e0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:50.164144 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4d674-1d7f-4f1d-842b-e2420944e0bb None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:50.164932 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4d674-1d7f-4f1d-842b-e2420944e0bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:50.165335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4d674-1d7f-4f1d-842b-e2420944e0bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:50.166176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d2614-8f6b-415b-9d28-3261f6893f43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:50.166905 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4d674-1d7f-4f1d-842b-e2420944e0bb None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:50.167778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d2614-8f6b-415b-9d28-3261f6893f43 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:50.179088 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 425/868] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:50.180151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4d674-1d7f-4f1d-842b-e2420944e0bb None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-036d2614-8f6b-415b-9d28-3261f6893f43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:50.180981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:50.181866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:50.182779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:50.182779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:50.182779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:50.182779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:50.182779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:50.182779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:50.182779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:50.182779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:50.182779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 444/869] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:50.189417 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d2614-8f6b-415b-9d28-3261f6893f43 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:50.291403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4d674-1d7f-4f1d-842b-e2420944e0bb None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:50.299960 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d2614-8f6b-415b-9d28-3261f6893f43 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:50.328221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20224d14-881e-43da-b105-af8329feb388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:50.328967 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20224d14-881e-43da-b105-af8329feb388 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:50.329093 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20224d14-881e-43da-b105-af8329feb388 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:50.329363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20224d14-881e-43da-b105-af8329feb388 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:50.330022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20224d14-881e-43da-b105-af8329feb388 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:50.331387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20224d14-881e-43da-b105-af8329feb388 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:50.337565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b41700-6b86-4416-bfcb-18e872dfcd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:50.338345 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b41700-6b86-4416-bfcb-18e872dfcd5e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:50.338803 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b41700-6b86-4416-bfcb-18e872dfcd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:50.339178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b41700-6b86-4416-bfcb-18e872dfcd5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:50.339965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b41700-6b86-4416-bfcb-18e872dfcd5e None admin] RBAC: Adding query filter params (role.id=5c7e1f557ae04250a01db783f3c27a5f, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=0212eb5841a04075a0fd7ba853231267) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:50.340261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b41700-6b86-4416-bfcb-18e872dfcd5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:50.341836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b41700-6b86-4416-bfcb-18e872dfcd5e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:50.346811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 426/870] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Sep 12 23:15:50 2022] POST /identity/v3/services => generated 269 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:15:50.347731 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20224d14-881e-43da-b105-af8329feb388 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:50.355680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 445/871] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:50 2022] GET /identity/v3/role_assignments?user.id=0212eb5841a04075a0fd7ba853231267&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=5c7e1f557ae04250a01db783f3c27a5f => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:50.355680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836d8d5-5226-4b0a-9e10-fb019676a948 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:50.355680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836d8d5-5226-4b0a-9e10-fb019676a948 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:50.355680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836d8d5-5226-4b0a-9e10-fb019676a948 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:50.355680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 446/872] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:50.362082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1baad-dbbe-4700-b2c6-da835e35ed84 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:50.362417 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1baad-dbbe-4700-b2c6-da835e35ed84 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:50.362695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1baad-dbbe-4700-b2c6-da835e35ed84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:50.384696 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c8f1baad-dbbe-4700-b2c6-da835e35ed84 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:50.385895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c8f1baad-dbbe-4700-b2c6-da835e35ed84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:50.410309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:50.424259 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:50.506638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20224d14-881e-43da-b105-af8329feb388 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:50.511767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 447/873] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:50 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:15:50.554022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ea947-2a24-47da-857a-13aa12c04529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:50.554655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ea947-2a24-47da-857a-13aa12c04529 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:50.554934 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ea947-2a24-47da-857a-13aa12c04529 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:50.555201 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ea947-2a24-47da-857a-13aa12c04529 None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:50.555923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ea947-2a24-47da-857a-13aa12c04529 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:50.556225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ea947-2a24-47da-857a-13aa12c04529 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:50.557658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ea947-2a24-47da-857a-13aa12c04529 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:50.572377 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 427/874] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:50.578379 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1baad-dbbe-4700-b2c6-da835e35ed84 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:50.688581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1baad-dbbe-4700-b2c6-da835e35ed84 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:50.724207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ca06a-f6ea-43a7-8c20-414252d68759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:50.725534 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ca06a-f6ea-43a7-8c20-414252d68759 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:50.725981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ca06a-f6ea-43a7-8c20-414252d68759 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:50.726398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ca06a-f6ea-43a7-8c20-414252d68759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:50.727266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ca06a-f6ea-43a7-8c20-414252d68759 None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=c1f6bb09fd7f4905a1693ef7cb6f088e) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:50.727764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ca06a-f6ea-43a7-8c20-414252d68759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:50.729329 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ca06a-f6ea-43a7-8c20-414252d68759 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:50.735827 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 448/875] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:50 2022] GET /identity/v3/role_assignments?user.id=c1f6bb09fd7f4905a1693ef7cb6f088e&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=23538dc4c770454e80dd11ed8e498539 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:50.791283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ea947-2a24-47da-857a-13aa12c04529 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:50.902696 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ea947-2a24-47da-857a-13aa12c04529 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:50.906796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:50.949387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47d805-51ca-4c77-8a9b-77f902bca46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:50.949975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47d805-51ca-4c77-8a9b-77f902bca46f None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:50.950256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47d805-51ca-4c77-8a9b-77f902bca46f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:50.950524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47d805-51ca-4c77-8a9b-77f902bca46f None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:50.951182 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47d805-51ca-4c77-8a9b-77f902bca46f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:50.952704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47d805-51ca-4c77-8a9b-77f902bca46f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:50.969170 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 428/876] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Sep 12 23:15:50 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:15:52.664608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3244f9e-ef85-4c7d-979e-4aacf041ada2 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:52.665135 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3244f9e-ef85-4c7d-979e-4aacf041ada2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:52.665135 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3244f9e-ef85-4c7d-979e-4aacf041ada2 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:52.666793 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 449/877] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:52.672218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bad706-3420-40b2-bae7-eecc0e6a5f41 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:52.672616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bad706-3420-40b2-bae7-eecc0e6a5f41 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:52.672781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bad706-3420-40b2-bae7-eecc0e6a5f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:52.692802 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c3bad706-3420-40b2-bae7-eecc0e6a5f41 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:52.694118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c3bad706-3420-40b2-bae7-eecc0e6a5f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:52.739984 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d40f2-3977-4447-bba2-955d3e9e5f87 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:52.740289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d40f2-3977-4447-bba2-955d3e9e5f87 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:52.740516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d40f2-3977-4447-bba2-955d3e9e5f87 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:52.742635 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 450/878] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:52.747500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4bfeb-1372-42ea-a916-90a5a23bbf61 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:52.747824 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4bfeb-1372-42ea-a916-90a5a23bbf61 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:52.748099 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4bfeb-1372-42ea-a916-90a5a23bbf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:52.755310 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:52.768209 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ffc4bfeb-1372-42ea-a916-90a5a23bbf61 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:52.769389 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ffc4bfeb-1372-42ea-a916-90a5a23bbf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:52.771510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 429/879] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:15:52.840727 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 451/880] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:15:52.979793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce53ac-a469-4555-b879-f584118147f2 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:52.980128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce53ac-a469-4555-b879-f584118147f2 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:52.980360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce53ac-a469-4555-b879-f584118147f2 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:52.981949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 430/881] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:52.987181 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3de3ee-5c46-461e-ae9c-77ebf0c30300 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:52.987726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3de3ee-5c46-461e-ae9c-77ebf0c30300 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:52.988219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3de3ee-5c46-461e-ae9c-77ebf0c30300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:53.008076 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2a3de3ee-5c46-461e-ae9c-77ebf0c30300 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:53.009660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2a3de3ee-5c46-461e-ae9c-77ebf0c30300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:53.055035 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce53ac-a469-4555-b879-f584118147f2 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:53.087229 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 452/882] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:15:53.116631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3de3ee-5c46-461e-ae9c-77ebf0c30300 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:53.153226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:53.166452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce53ac-a469-4555-b879-f584118147f2 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:53.188083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:53.202454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ef9ae-3e62-4ad8-bdde-69da94f5c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:53.203083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ef9ae-3e62-4ad8-bdde-69da94f5c25f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:53.203363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ef9ae-3e62-4ad8-bdde-69da94f5c25f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:53.203656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ef9ae-3e62-4ad8-bdde-69da94f5c25f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:53.204238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ef9ae-3e62-4ad8-bdde-69da94f5c25f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:53.205600 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ef9ae-3e62-4ad8-bdde-69da94f5c25f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-ad7ef9ae-3e62-4ad8-bdde-69da94f5c25f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:53.215249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:53.216282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:53.217198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 431/883] 10.222.0.27 () {56 vars in 1132 bytes} [Mon Sep 12 23:15:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:53.218893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d97323-a306-4e7b-8391-8c3756c73e16 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:53.218893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d97323-a306-4e7b-8391-8c3756c73e16 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:53.218893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d97323-a306-4e7b-8391-8c3756c73e16 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:53.223609 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 432/884] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:53.223609 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d97323-a306-4e7b-8391-8c3756c73e16 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:53.238538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3de3ee-5c46-461e-ae9c-77ebf0c30300 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:53.275647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ae0c9-7433-41b4-8d10-ea371d008281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:53.276337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ae0c9-7433-41b4-8d10-ea371d008281 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:53.276625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ae0c9-7433-41b4-8d10-ea371d008281 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:53.276930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ae0c9-7433-41b4-8d10-ea371d008281 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:53.289722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ae0c9-7433-41b4-8d10-ea371d008281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-038ae0c9-7433-41b4-8d10-ea371d008281 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:53.294767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:53.296078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:53.296957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:53.296957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:53.296957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:53.296957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:53.296957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:53.296957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:53.296957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:53.296957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:53.296957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:53.296957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:53.296957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:15:53.296957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:53.301620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 453/885] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Sep 12 23:15:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:15:53.302615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fa3c9-0dcd-4559-9a88-78856186fdbe None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:53.302899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fa3c9-0dcd-4559-9a88-78856186fdbe None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:53.303175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fa3c9-0dcd-4559-9a88-78856186fdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:53.323053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a48fa3c9-0dcd-4559-9a88-78856186fdbe None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:53.324284 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a48fa3c9-0dcd-4559-9a88-78856186fdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:53.329951 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d97323-a306-4e7b-8391-8c3756c73e16 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:53.366505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88669b-d1fa-4e0e-b7c6-db891f6293ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:53.367166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88669b-d1fa-4e0e-b7c6-db891f6293ec None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:53.367482 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88669b-d1fa-4e0e-b7c6-db891f6293ec None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:53.367793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88669b-d1fa-4e0e-b7c6-db891f6293ec None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:53.368423 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88669b-d1fa-4e0e-b7c6-db891f6293ec None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:53.368723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88669b-d1fa-4e0e-b7c6-db891f6293ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:53.370225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88669b-d1fa-4e0e-b7c6-db891f6293ec None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:53.379585 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 433/886] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Sep 12 23:15:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:53.379585 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88669b-d1fa-4e0e-b7c6-db891f6293ec None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:53.393076 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 454/887] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:15:53.393927 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fa3c9-0dcd-4559-9a88-78856186fdbe None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:53.399463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:53.450052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:53.491963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88669b-d1fa-4e0e-b7c6-db891f6293ec None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:53.498160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fa3c9-0dcd-4559-9a88-78856186fdbe None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:53.524970 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e01b9-55c3-48f1-a614-b8829047dcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:53.525575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e01b9-55c3-48f1-a614-b8829047dcbc None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:53.525854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e01b9-55c3-48f1-a614-b8829047dcbc None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:53.526138 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e01b9-55c3-48f1-a614-b8829047dcbc None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:53.526779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e01b9-55c3-48f1-a614-b8829047dcbc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:53.527087 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e01b9-55c3-48f1-a614-b8829047dcbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:53.528475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e01b9-55c3-48f1-a614-b8829047dcbc None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:53.534809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af6a2c-840a-4e70-957a-8cca5879971e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:53.535449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af6a2c-840a-4e70-957a-8cca5879971e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:53.535762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af6a2c-840a-4e70-957a-8cca5879971e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:53.536039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af6a2c-840a-4e70-957a-8cca5879971e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:53.536582 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 434/888] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Sep 12 23:15:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:53.536701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af6a2c-840a-4e70-957a-8cca5879971e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:53.538591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af6a2c-840a-4e70-957a-8cca5879971e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:53.540557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e01b9-55c3-48f1-a614-b8829047dcbc None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-63af6a2c-840a-4e70-957a-8cca5879971e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:53.548596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:53.549605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:53.550445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 455/889] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Sep 12 23:15:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:53.551242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af6a2c-840a-4e70-957a-8cca5879971e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:53.635869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:53.657121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e01b9-55c3-48f1-a614-b8829047dcbc None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:53.663475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af6a2c-840a-4e70-957a-8cca5879971e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:53.680749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:53.692251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6404d-078d-46a2-ba0e-1c7f894cf5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:53.692845 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6404d-078d-46a2-ba0e-1c7f894cf5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:53.693127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6404d-078d-46a2-ba0e-1c7f894cf5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:53.693394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6404d-078d-46a2-ba0e-1c7f894cf5d6 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:53.694651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6404d-078d-46a2-ba0e-1c7f894cf5d6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:53.694954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6404d-078d-46a2-ba0e-1c7f894cf5d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:53.696378 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6404d-078d-46a2-ba0e-1c7f894cf5d6 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:53.698144 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb39a12-28c7-4e80-98bd-e0fedd1cb1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:53.698761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb39a12-28c7-4e80-98bd-e0fedd1cb1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:53.699042 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb39a12-28c7-4e80-98bd-e0fedd1cb1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:53.699319 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb39a12-28c7-4e80-98bd-e0fedd1cb1d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:53.701271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 435/890] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Sep 12 23:15:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:53.702306 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6404d-078d-46a2-ba0e-1c7f894cf5d6 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:53.709423 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb39a12-28c7-4e80-98bd-e0fedd1cb1d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:53.710903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb39a12-28c7-4e80-98bd-e0fedd1cb1d6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4fb39a12-28c7-4e80-98bd-e0fedd1cb1d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:53.725059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:53.725975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:53.726862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:53.726862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:53.726862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:53.726862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 456/891] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:53.727083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb39a12-28c7-4e80-98bd-e0fedd1cb1d6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:53.802891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6404d-078d-46a2-ba0e-1c7f894cf5d6 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:53.826237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb39a12-28c7-4e80-98bd-e0fedd1cb1d6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:53.835283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896e00b-86d4-4009-815e-282c6664db2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:53.835913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896e00b-86d4-4009-815e-282c6664db2a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:53.836196 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896e00b-86d4-4009-815e-282c6664db2a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:53.836471 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896e00b-86d4-4009-815e-282c6664db2a None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:53.837090 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896e00b-86d4-4009-815e-282c6664db2a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:53.837392 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896e00b-86d4-4009-815e-282c6664db2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:53.838758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896e00b-86d4-4009-815e-282c6664db2a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:53.843273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 436/892] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Sep 12 23:15:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:53.844075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896e00b-86d4-4009-815e-282c6664db2a None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:53.857541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:53.858878 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c702337-c999-43a0-8dce-89d2e87a77d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:53.859706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c702337-c999-43a0-8dce-89d2e87a77d8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:53.859992 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c702337-c999-43a0-8dce-89d2e87a77d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:53.860267 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c702337-c999-43a0-8dce-89d2e87a77d8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:53.860843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c702337-c999-43a0-8dce-89d2e87a77d8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:53.862209 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c702337-c999-43a0-8dce-89d2e87a77d8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8c702337-c999-43a0-8dce-89d2e87a77d8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:53.871776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:53.872809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:53.873636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 457/893] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:15:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:53.874452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c702337-c999-43a0-8dce-89d2e87a77d8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:53.907922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:53.948379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896e00b-86d4-4009-815e-282c6664db2a None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:53.975867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c702337-c999-43a0-8dce-89d2e87a77d8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:53.980917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b133c7-f818-4da0-a916-99b4811deecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:53.981519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b133c7-f818-4da0-a916-99b4811deecd None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:53.981792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b133c7-f818-4da0-a916-99b4811deecd None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:53.982058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b133c7-f818-4da0-a916-99b4811deecd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:53.982747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b133c7-f818-4da0-a916-99b4811deecd None admin] RBAC: Adding query filter params (interface=public, service_id=0ae0e52032e840018c3c0186e5192dc0, region_id=RegionOne) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:53.983052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b133c7-f818-4da0-a916-99b4811deecd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:53.984437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b133c7-f818-4da0-a916-99b4811deecd None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:53.993714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 437/894] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Sep 12 23:15:53 2022] GET /identity/v3/endpoints?service_id=0ae0e52032e840018c3c0186e5192dc0&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:53.993714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b133c7-f818-4da0-a916-99b4811deecd None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:54.008820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07f14e-6270-499a-8de9-01bb35a620d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:54.009424 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07f14e-6270-499a-8de9-01bb35a620d0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:54.009702 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07f14e-6270-499a-8de9-01bb35a620d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:54.009977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07f14e-6270-499a-8de9-01bb35a620d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:54.010606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07f14e-6270-499a-8de9-01bb35a620d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:54.010914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07f14e-6270-499a-8de9-01bb35a620d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:54.012294 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07f14e-6270-499a-8de9-01bb35a620d0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:54.023345 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 458/895] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:54.024145 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07f14e-6270-499a-8de9-01bb35a620d0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:54.065512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:54.094689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b133c7-f818-4da0-a916-99b4811deecd None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:54.113876 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:54.121494 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07f14e-6270-499a-8de9-01bb35a620d0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:54.126651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827fc4c-685e-4fff-af3b-6c9338efb139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:54.127258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827fc4c-685e-4fff-af3b-6c9338efb139 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:54.127538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827fc4c-685e-4fff-af3b-6c9338efb139 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:54.127844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827fc4c-685e-4fff-af3b-6c9338efb139 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:54.128471 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827fc4c-685e-4fff-af3b-6c9338efb139 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:54.128770 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827fc4c-685e-4fff-af3b-6c9338efb139 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:54.130131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827fc4c-685e-4fff-af3b-6c9338efb139 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:54.135071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 438/896] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Sep 12 23:15:53 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:54.135879 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827fc4c-685e-4fff-af3b-6c9338efb139 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:54.157804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c3c3b-a17a-435f-b3d0-8a71d4d32e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:54.158912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c3c3b-a17a-435f-b3d0-8a71d4d32e47 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:54.159195 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c3c3b-a17a-435f-b3d0-8a71d4d32e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:54.159466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c3c3b-a17a-435f-b3d0-8a71d4d32e47 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:54.160170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c3c3b-a17a-435f-b3d0-8a71d4d32e47 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:54.160476 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c3c3b-a17a-435f-b3d0-8a71d4d32e47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:54.162176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c3c3b-a17a-435f-b3d0-8a71d4d32e47 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:54.169314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 459/897] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:15:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:54.170188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c3c3b-a17a-435f-b3d0-8a71d4d32e47 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:54.248708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827fc4c-685e-4fff-af3b-6c9338efb139 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:54.284868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c3c3b-a17a-435f-b3d0-8a71d4d32e47 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:54.286698 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c459e-7c95-4cba-8621-1489e80ab64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:54.287321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c459e-7c95-4cba-8621-1489e80ab64e None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:54.287662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c459e-7c95-4cba-8621-1489e80ab64e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:54.287948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c459e-7c95-4cba-8621-1489e80ab64e None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:54.288558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c459e-7c95-4cba-8621-1489e80ab64e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:54.289963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c459e-7c95-4cba-8621-1489e80ab64e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:54.291974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:54.297309 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 439/898] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Sep 12 23:15:54 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:15:54.299076 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c459e-7c95-4cba-8621-1489e80ab64e None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:54.331522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf46a4-5215-4a68-b1bf-c4236bb0e899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:54.332219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf46a4-5215-4a68-b1bf-c4236bb0e899 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:54.332500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf46a4-5215-4a68-b1bf-c4236bb0e899 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:54.332777 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf46a4-5215-4a68-b1bf-c4236bb0e899 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:54.345519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf46a4-5215-4a68-b1bf-c4236bb0e899 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:54.346682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf46a4-5215-4a68-b1bf-c4236bb0e899 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-92bf46a4-5215-4a68-b1bf-c4236bb0e899 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:54.363163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:54.364325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:54.365695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:54.365695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:54.365695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:54.365695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:54.365695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:54.365695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:54.365695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:54.365695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:54.365695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:15:54.365695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:54.365695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 460/899] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Sep 12 23:15:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:54.370325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf46a4-5215-4a68-b1bf-c4236bb0e899 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:54.393878 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:54.464900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c459e-7c95-4cba-8621-1489e80ab64e None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:54.506143 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760324f7-8a6b-4046-823d-3e90bd7636b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:54.506757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760324f7-8a6b-4046-823d-3e90bd7636b0 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:54.507043 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760324f7-8a6b-4046-823d-3e90bd7636b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:54.507322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760324f7-8a6b-4046-823d-3e90bd7636b0 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:54.508004 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760324f7-8a6b-4046-823d-3e90bd7636b0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:54.508310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760324f7-8a6b-4046-823d-3e90bd7636b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:54.509118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf46a4-5215-4a68-b1bf-c4236bb0e899 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:54.509704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760324f7-8a6b-4046-823d-3e90bd7636b0 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:54.515536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 440/900] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:15:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:54.523164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760324f7-8a6b-4046-823d-3e90bd7636b0 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:54.556227 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cfe7a-e2c6-4c97-8f20-38a980758fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:54.556842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cfe7a-e2c6-4c97-8f20-38a980758fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:54.557125 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cfe7a-e2c6-4c97-8f20-38a980758fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:54.557400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cfe7a-e2c6-4c97-8f20-38a980758fd6 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:54.558132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cfe7a-e2c6-4c97-8f20-38a980758fd6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:54.558439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cfe7a-e2c6-4c97-8f20-38a980758fd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:54.559887 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cfe7a-e2c6-4c97-8f20-38a980758fd6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:54.588893 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 461/901] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Sep 12 23:15:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:54.589605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cfe7a-e2c6-4c97-8f20-38a980758fd6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:54.619997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:54.646103 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760324f7-8a6b-4046-823d-3e90bd7636b0 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:54.685450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:54.691247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd4482-9362-4564-8f72-80a1305053d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:54.691892 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd4482-9362-4564-8f72-80a1305053d6 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:54.692177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd4482-9362-4564-8f72-80a1305053d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:54.692452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd4482-9362-4564-8f72-80a1305053d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:54.693156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd4482-9362-4564-8f72-80a1305053d6 None admin] RBAC: Adding query filter params (interface=public, service_id=67a1d185372f44a685cfae6eab926464, region_id=RegionOne) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:54.693464 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd4482-9362-4564-8f72-80a1305053d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:54.695380 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd4482-9362-4564-8f72-80a1305053d6 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:54.701286 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 441/902] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Sep 12 23:15:54 2022] GET /identity/v3/endpoints?service_id=67a1d185372f44a685cfae6eab926464&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:54.708212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd4482-9362-4564-8f72-80a1305053d6 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:54.721744 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cfe7a-e2c6-4c97-8f20-38a980758fd6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:54.770408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf907bf-ae2e-4520-b578-c09cf3152067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:54.772043 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf907bf-ae2e-4520-b578-c09cf3152067 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:54.772570 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf907bf-ae2e-4520-b578-c09cf3152067 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:54.773280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf907bf-ae2e-4520-b578-c09cf3152067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:54.820632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd4482-9362-4564-8f72-80a1305053d6 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:54.848778 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:54.855169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf907bf-ae2e-4520-b578-c09cf3152067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:54.856663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf907bf-ae2e-4520-b578-c09cf3152067 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:54.859007 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257547a-2c66-4cd2-ad4c-f12b8653c667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:54.859922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257547a-2c66-4cd2-ad4c-f12b8653c667 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:54.860404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257547a-2c66-4cd2-ad4c-f12b8653c667 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:54.860902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257547a-2c66-4cd2-ad4c-f12b8653c667 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:54.861703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257547a-2c66-4cd2-ad4c-f12b8653c667 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:54.863274 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257547a-2c66-4cd2-ad4c-f12b8653c667 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:54.870097 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 442/903] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Sep 12 23:15:54 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:15:54.963947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:54.968694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 462/904] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:15:54 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:15:54.974376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257547a-2c66-4cd2-ad4c-f12b8653c667 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:55.092637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257547a-2c66-4cd2-ad4c-f12b8653c667 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:55.129260 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f7bdc-49dc-4d52-b1b2-d14dbe5a0ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:55.129896 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f7bdc-49dc-4d52-b1b2-d14dbe5a0ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:55.130177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f7bdc-49dc-4d52-b1b2-d14dbe5a0ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:55.130453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f7bdc-49dc-4d52-b1b2-d14dbe5a0ad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:55.142490 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f7bdc-49dc-4d52-b1b2-d14dbe5a0ad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:55.144054 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f7bdc-49dc-4d52-b1b2-d14dbe5a0ad9 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-6c1f7bdc-49dc-4d52-b1b2-d14dbe5a0ad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:55.159763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:55.160709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:55.161591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:55.161591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:55.161591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:55.161591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 443/905] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:55.166801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf907bf-ae2e-4520-b578-c09cf3152067 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:55.275597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf907bf-ae2e-4520-b578-c09cf3152067 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:55.312029 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b21e8-15b2-4970-bf59-941bee7ad375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:55.312698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b21e8-15b2-4970-bf59-941bee7ad375 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:55.312985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b21e8-15b2-4970-bf59-941bee7ad375 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:55.313260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b21e8-15b2-4970-bf59-941bee7ad375 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:55.313900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b21e8-15b2-4970-bf59-941bee7ad375 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:55.314204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b21e8-15b2-4970-bf59-941bee7ad375 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:55.315579 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b21e8-15b2-4970-bf59-941bee7ad375 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:55.328115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 463/906] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:55.333311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f7bdc-49dc-4d52-b1b2-d14dbe5a0ad9 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:55.382263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:55.440099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f7bdc-49dc-4d52-b1b2-d14dbe5a0ad9 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:55.476226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc7520-d6f6-4d8d-8620-32ac72c2597b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:55.476861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc7520-d6f6-4d8d-8620-32ac72c2597b None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:55.477146 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc7520-d6f6-4d8d-8620-32ac72c2597b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:55.477418 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc7520-d6f6-4d8d-8620-32ac72c2597b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:55.484490 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc7520-d6f6-4d8d-8620-32ac72c2597b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:55.485941 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc7520-d6f6-4d8d-8620-32ac72c2597b None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-8adc7520-d6f6-4d8d-8620-32ac72c2597b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:15:55.497456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:55.499386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:55.501533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:15:55.501533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:15:55.501533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:15:55.501533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:15:55.501533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:15:55.501533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:15:55.501533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:15:55.501533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:55.501533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 444/907] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:15:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:55.508659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b21e8-15b2-4970-bf59-941bee7ad375 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:55.546086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:55.616899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b21e8-15b2-4970-bf59-941bee7ad375 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:55.654114 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f55e17-a495-4e96-8a78-e941c5d30665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:55.654701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f55e17-a495-4e96-8a78-e941c5d30665 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:55.654981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f55e17-a495-4e96-8a78-e941c5d30665 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:55.655256 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f55e17-a495-4e96-8a78-e941c5d30665 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:55.656025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f55e17-a495-4e96-8a78-e941c5d30665 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:55.656337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f55e17-a495-4e96-8a78-e941c5d30665 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:55.658075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f55e17-a495-4e96-8a78-e941c5d30665 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:55.670814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 464/908] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:15:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:55.676741 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc7520-d6f6-4d8d-8620-32ac72c2597b None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:55.786355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc7520-d6f6-4d8d-8620-32ac72c2597b None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:55.790220 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:55.825595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d98f3e-cbc5-47b3-85a4-461bdc09f4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:55.826574 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98f3e-cbc5-47b3-85a4-461bdc09f4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:55.827075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98f3e-cbc5-47b3-85a4-461bdc09f4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:55.827523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98f3e-cbc5-47b3-85a4-461bdc09f4f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:55.828450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d98f3e-cbc5-47b3-85a4-461bdc09f4f9 None admin] RBAC: Adding query filter params (role.id=9940e26019d341f4a9af2fa399a02bc8, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=0212eb5841a04075a0fd7ba853231267) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:55.828929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d98f3e-cbc5-47b3-85a4-461bdc09f4f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:55.830524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d98f3e-cbc5-47b3-85a4-461bdc09f4f9 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:55.837022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 445/909] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:15:55 2022] GET /identity/v3/role_assignments?user.id=0212eb5841a04075a0fd7ba853231267&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=9940e26019d341f4a9af2fa399a02bc8 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:56.380351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b436f63-19a2-4ed8-b4cb-a7ad952e352f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:56.380835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b436f63-19a2-4ed8-b4cb-a7ad952e352f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:56.380835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b436f63-19a2-4ed8-b4cb-a7ad952e352f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:56.382928 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 465/910] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:56.388438 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78d7fd-cf0d-426d-929d-211d4d20fe27 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:56.388725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78d7fd-cf0d-426d-929d-211d4d20fe27 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:56.389001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78d7fd-cf0d-426d-929d-211d4d20fe27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:56.407838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-3f78d7fd-cf0d-426d-929d-211d4d20fe27 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:56.409023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3f78d7fd-cf0d-426d-929d-211d4d20fe27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:56.483724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 446/911] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:15:56.606371 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe3bcb-7008-4dce-b510-159d8a30872c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:56.606718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe3bcb-7008-4dce-b510-159d8a30872c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:56.606932 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe3bcb-7008-4dce-b510-159d8a30872c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:56.608558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 466/912] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:56.613825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557f43a-b27d-4602-92b5-e4dcd084adb5 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:56.614113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557f43a-b27d-4602-92b5-e4dcd084adb5 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:56.614390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557f43a-b27d-4602-92b5-e4dcd084adb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:56.633730 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-d557f43a-b27d-4602-92b5-e4dcd084adb5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:56.634848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d557f43a-b27d-4602-92b5-e4dcd084adb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:56.706979 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 447/913] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:15:56.747597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe3bcb-7008-4dce-b510-159d8a30872c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:56.852768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:56.860958 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe3bcb-7008-4dce-b510-159d8a30872c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:56.897627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3960038-c06a-407d-9520-d46b5b1af91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:56.898159 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3960038-c06a-407d-9520-d46b5b1af91a None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:56.898439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3960038-c06a-407d-9520-d46b5b1af91a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:56.898710 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3960038-c06a-407d-9520-d46b5b1af91a None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:56.899372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3960038-c06a-407d-9520-d46b5b1af91a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:56.900761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3960038-c06a-407d-9520-d46b5b1af91a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:56.920620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 467/914] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Sep 12 23:15:56 2022] POST /identity/v3/services => generated 241 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:15:57.001784 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557f43a-b27d-4602-92b5-e4dcd084adb5 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:57.033451 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:57.092032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785c7fc-caa8-465e-8bd2-7d597f09c65c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:57.092361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785c7fc-caa8-465e-8bd2-7d597f09c65c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:57.092576 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785c7fc-caa8-465e-8bd2-7d597f09c65c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:57.094381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 468/915] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:57.099294 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3e6ee-3fdf-41df-8110-d06374512ca9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:57.099595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3e6ee-3fdf-41df-8110-d06374512ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:57.099868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3e6ee-3fdf-41df-8110-d06374512ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:57.116951 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557f43a-b27d-4602-92b5-e4dcd084adb5 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:57.123521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dfb3e6ee-3fdf-41df-8110-d06374512ca9 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:57.124829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dfb3e6ee-3fdf-41df-8110-d06374512ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:57.166673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe395f-3503-4af1-b060-1ad14d3d98da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:57.167266 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe395f-3503-4af1-b060-1ad14d3d98da None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:57.167548 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe395f-3503-4af1-b060-1ad14d3d98da None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:57.167859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe395f-3503-4af1-b060-1ad14d3d98da None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:57.168664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe395f-3503-4af1-b060-1ad14d3d98da None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:57.170164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe395f-3503-4af1-b060-1ad14d3d98da None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-4efe395f-3503-4af1-b060-1ad14d3d98da None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:57.180102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:57.181102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:57.182167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 448/916] 10.222.0.27 () {56 vars in 1132 bytes} [Mon Sep 12 23:15:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:57.187437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe395f-3503-4af1-b060-1ad14d3d98da None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:57.214981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 469/917] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:15:57.285216 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:57.297786 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe395f-3503-4af1-b060-1ad14d3d98da None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:57.332534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa2c26-7fe4-4c4e-8f99-b4eb38c7971b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:57.333143 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa2c26-7fe4-4c4e-8f99-b4eb38c7971b None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:57.333422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa2c26-7fe4-4c4e-8f99-b4eb38c7971b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:57.333694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa2c26-7fe4-4c4e-8f99-b4eb38c7971b None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:57.334321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa2c26-7fe4-4c4e-8f99-b4eb38c7971b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:57.335149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa2c26-7fe4-4c4e-8f99-b4eb38c7971b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:57.336681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa2c26-7fe4-4c4e-8f99-b4eb38c7971b None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:57.341860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 449/918] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Sep 12 23:15:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:57.347585 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3e6ee-3fdf-41df-8110-d06374512ca9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:57.437890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:57.454911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3e6ee-3fdf-41df-8110-d06374512ca9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:57.478229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa2c26-7fe4-4c4e-8f99-b4eb38c7971b None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:57.501473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425f668-3e92-44eb-8af7-de9b4b403366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:57.502493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425f668-3e92-44eb-8af7-de9b4b403366 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:57.502993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425f668-3e92-44eb-8af7-de9b4b403366 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:57.503483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425f668-3e92-44eb-8af7-de9b4b403366 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:57.504441 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425f668-3e92-44eb-8af7-de9b4b403366 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:57.505055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425f668-3e92-44eb-8af7-de9b4b403366 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:57.506721 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425f668-3e92-44eb-8af7-de9b4b403366 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:57.512641 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 470/919] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Sep 12 23:15:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:57.523652 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425f668-3e92-44eb-8af7-de9b4b403366 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:57.595104 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa2c26-7fe4-4c4e-8f99-b4eb38c7971b None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:57.631167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425f668-3e92-44eb-8af7-de9b4b403366 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:57.639719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205c785-060d-4413-a9be-6ded70b69532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:57.640379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205c785-060d-4413-a9be-6ded70b69532 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:57.640656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205c785-060d-4413-a9be-6ded70b69532 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:57.642845 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205c785-060d-4413-a9be-6ded70b69532 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:57.643297 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205c785-060d-4413-a9be-6ded70b69532 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:57.644717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205c785-060d-4413-a9be-6ded70b69532 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-6205c785-060d-4413-a9be-6ded70b69532 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:57.654611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:57.655648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:57.656475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 450/920] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:15:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:57.661632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205c785-060d-4413-a9be-6ded70b69532 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:57.676722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cc4b0-137d-401a-9066-9b441db43aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:57.677746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cc4b0-137d-401a-9066-9b441db43aac None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:57.678207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cc4b0-137d-401a-9066-9b441db43aac None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:57.679291 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cc4b0-137d-401a-9066-9b441db43aac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:57.680118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1cc4b0-137d-401a-9066-9b441db43aac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:57.681660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1cc4b0-137d-401a-9066-9b441db43aac None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:57.685284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:57.712466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 471/921] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:15:57 2022] POST /identity/v3/endpoints => generated 347 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:15:57.794459 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205c785-060d-4413-a9be-6ded70b69532 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:57.840586 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e50ce-8627-4612-8c9a-11e90578e7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:57.841261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e50ce-8627-4612-8c9a-11e90578e7bf None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:57.841545 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e50ce-8627-4612-8c9a-11e90578e7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:57.841814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e50ce-8627-4612-8c9a-11e90578e7bf None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:57.842505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e50ce-8627-4612-8c9a-11e90578e7bf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:57.842810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e50ce-8627-4612-8c9a-11e90578e7bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:57.844353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e50ce-8627-4612-8c9a-11e90578e7bf None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:57.850989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 451/922] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:15:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:57.856756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cc4b0-137d-401a-9066-9b441db43aac None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:57.875965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:57.971632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cc4b0-137d-401a-9066-9b441db43aac None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:58.010380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43852e4-9c76-470a-9df9-7cbdf376e7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:58.011625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43852e4-9c76-470a-9df9-7cbdf376e7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:58.012119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43852e4-9c76-470a-9df9-7cbdf376e7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:58.012627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43852e4-9c76-470a-9df9-7cbdf376e7a3 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:58.013494 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43852e4-9c76-470a-9df9-7cbdf376e7a3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:58.014007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43852e4-9c76-470a-9df9-7cbdf376e7a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:58.015690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43852e4-9c76-470a-9df9-7cbdf376e7a3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:58.022724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 472/923] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Sep 12 23:15:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:58.031602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e50ce-8627-4612-8c9a-11e90578e7bf None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:58.109511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:58.114647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f078a18-35a9-48b7-b7ea-60832c33596c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:58.114937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f078a18-35a9-48b7-b7ea-60832c33596c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:58.115217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f078a18-35a9-48b7-b7ea-60832c33596c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:58.117491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 473/924] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:15:58.123095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ef670-0e5a-47dd-9a63-33aa72e319e9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:58.123642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ef670-0e5a-47dd-9a63-33aa72e319e9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:58.124111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ef670-0e5a-47dd-9a63-33aa72e319e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:58.136980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e50ce-8627-4612-8c9a-11e90578e7bf None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:58.145716 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-139ef670-0e5a-47dd-9a63-33aa72e319e9 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:58.148154 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-139ef670-0e5a-47dd-9a63-33aa72e319e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:58.176292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464e758-83e3-463b-ad6a-7bb94a6334a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:58.177281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464e758-83e3-463b-ad6a-7bb94a6334a5 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:58.177831 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464e758-83e3-463b-ad6a-7bb94a6334a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:58.178933 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464e758-83e3-463b-ad6a-7bb94a6334a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:58.179986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464e758-83e3-463b-ad6a-7bb94a6334a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:58.181626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464e758-83e3-463b-ad6a-7bb94a6334a5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:58.206161 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 452/925] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:15:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:15:58.230578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:58.243200 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 474/926] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:15:58.530463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464e758-83e3-463b-ad6a-7bb94a6334a5 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:58.643628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464e758-83e3-463b-ad6a-7bb94a6334a5 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:58.674405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:58.680178 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c3e02-5849-495e-a192-91b4e5f9bce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:58.681349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c3e02-5849-495e-a192-91b4e5f9bce4 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:58.681984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c3e02-5849-495e-a192-91b4e5f9bce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:58.682514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c3e02-5849-495e-a192-91b4e5f9bce4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:58.694619 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c3e02-5849-495e-a192-91b4e5f9bce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-b42c3e02-5849-495e-a192-91b4e5f9bce4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:15:58.699983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:58.700912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:58.701777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:58.701777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:58.701777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:58.701777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:58.701777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:15:58.701777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:15:58.701777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:15:58.701777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:15:58.701777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:15:58.701777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:15:58.701777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:15:58.701777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:58.701777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 453/927] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Sep 12 23:15:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:58.709094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ef670-0e5a-47dd-9a63-33aa72e319e9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:58.827509 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ef670-0e5a-47dd-9a63-33aa72e319e9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:58.866625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f52212-58bc-49e4-aef4-635f100cad11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:58.867230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f52212-58bc-49e4-aef4-635f100cad11 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:58.867506 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f52212-58bc-49e4-aef4-635f100cad11 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:58.867812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f52212-58bc-49e4-aef4-635f100cad11 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:58.868466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f52212-58bc-49e4-aef4-635f100cad11 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:58.868764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f52212-58bc-49e4-aef4-635f100cad11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:58.870112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f52212-58bc-49e4-aef4-635f100cad11 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:58.878620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 475/928] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Sep 12 23:15:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:58.884229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c3e02-5849-495e-a192-91b4e5f9bce4 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:58.994859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c3e02-5849-495e-a192-91b4e5f9bce4 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:59.030560 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a0198-f45b-4887-a738-146a91a91b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:59.031192 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a0198-f45b-4887-a738-146a91a91b04 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:59.031472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a0198-f45b-4887-a738-146a91a91b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:59.031777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a0198-f45b-4887-a738-146a91a91b04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:59.043262 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a0198-f45b-4887-a738-146a91a91b04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:59.044765 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a0198-f45b-4887-a738-146a91a91b04 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-748a0198-f45b-4887-a738-146a91a91b04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:59.059605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:59.060595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:59.061494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:59.061494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:59.061494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:59.061494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 454/929] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:59.067805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f52212-58bc-49e4-aef4-635f100cad11 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:59.138041 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:59.192063 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f52212-58bc-49e4-aef4-635f100cad11 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:59.230377 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60354db6-4872-497f-8a97-689f4258d318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:59.231009 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60354db6-4872-497f-8a97-689f4258d318 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:59.231286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60354db6-4872-497f-8a97-689f4258d318 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:59.231558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60354db6-4872-497f-8a97-689f4258d318 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:59.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60354db6-4872-497f-8a97-689f4258d318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:59.232546 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60354db6-4872-497f-8a97-689f4258d318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:59.233955 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60354db6-4872-497f-8a97-689f4258d318 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:59.248178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 476/930] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:15:59.257549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a0198-f45b-4887-a738-146a91a91b04 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:59.307061 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:59.364831 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a0198-f45b-4887-a738-146a91a91b04 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:59.378194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ba262-4126-405b-a92f-16b5525ae4b0 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:59.378484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ba262-4126-405b-a92f-16b5525ae4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:59.378756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ba262-4126-405b-a92f-16b5525ae4b0 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:59.380245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 477/931] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:15:59.387600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19645-ae18-4a83-9299-4450097ebecb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:59.387600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19645-ae18-4a83-9299-4450097ebecb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:59.387600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19645-ae18-4a83-9299-4450097ebecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:59.406311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e7a9c-dd44-4023-a5d6-2dfbff52cf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:59.407228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e7a9c-dd44-4023-a5d6-2dfbff52cf3b None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:59.407948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e7a9c-dd44-4023-a5d6-2dfbff52cf3b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:59.408360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e7a9c-dd44-4023-a5d6-2dfbff52cf3b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:59.408581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-eaa19645-ae18-4a83-9299-4450097ebecb None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:15:59.409876 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eaa19645-ae18-4a83-9299-4450097ebecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:15:59.418226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e7a9c-dd44-4023-a5d6-2dfbff52cf3b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:59.420303 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e7a9c-dd44-4023-a5d6-2dfbff52cf3b None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-124e7a9c-dd44-4023-a5d6-2dfbff52cf3b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:15:59.435618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:59.436780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:59.437947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:59.437947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:59.437947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:15:59.437947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:15:59.437947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:15:59.437947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:15:59.437947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:15:59.437947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:15:59.437947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:15:59.437947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:15:59.440774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 455/932] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Sep 12 23:15:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:59.451729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e7a9c-dd44-4023-a5d6-2dfbff52cf3b None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:59.490538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 478/933] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:15:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:15:59.564119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e7a9c-dd44-4023-a5d6-2dfbff52cf3b None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:59.567271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:59.602629 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871a600-e736-4382-9e55-e4ce9cb43224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:59.603241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871a600-e736-4382-9e55-e4ce9cb43224 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:59.603528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871a600-e736-4382-9e55-e4ce9cb43224 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:59.603834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871a600-e736-4382-9e55-e4ce9cb43224 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:59.604562 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871a600-e736-4382-9e55-e4ce9cb43224 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:59.604867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871a600-e736-4382-9e55-e4ce9cb43224 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:59.606307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871a600-e736-4382-9e55-e4ce9cb43224 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:59.624649 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 456/934] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Sep 12 23:15:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:15:59.630578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa19645-ae18-4a83-9299-4450097ebecb None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:59.681582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:59.734823 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa19645-ae18-4a83-9299-4450097ebecb None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:59.770144 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9fbf2-8be0-4cd9-bb03-088fc60be1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:59.770775 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9fbf2-8be0-4cd9-bb03-088fc60be1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:59.771056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9fbf2-8be0-4cd9-bb03-088fc60be1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:59.771334 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9fbf2-8be0-4cd9-bb03-088fc60be1b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:59.781600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9fbf2-8be0-4cd9-bb03-088fc60be1b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:59.783290 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9fbf2-8be0-4cd9-bb03-088fc60be1b7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-19b9fbf2-8be0-4cd9-bb03-088fc60be1b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:15:59.800683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:59.801650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:15:59.802533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:15:59.802533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:15:59.802533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:59.802533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 479/935] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:59.807858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871a600-e736-4382-9e55-e4ce9cb43224 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:59.812433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9fbf2-8be0-4cd9-bb03-088fc60be1b7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:15:59.934601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:15:59.937523 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9fbf2-8be0-4cd9-bb03-088fc60be1b7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:59.949552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871a600-e736-4382-9e55-e4ce9cb43224 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:15:59.974091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990f9b7-20b9-473a-9c48-ec1cc91dcc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:59.975900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990f9b7-20b9-473a-9c48-ec1cc91dcc81 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:59.975900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990f9b7-20b9-473a-9c48-ec1cc91dcc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:59.976196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990f9b7-20b9-473a-9c48-ec1cc91dcc81 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:59.976896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990f9b7-20b9-473a-9c48-ec1cc91dcc81 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:59.978403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990f9b7-20b9-473a-9c48-ec1cc91dcc81 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c990f9b7-20b9-473a-9c48-ec1cc91dcc81 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:15:59.990365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:15:59.991801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:15:59.992788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:15:59.992788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:15:59.992788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:15:59.992788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:15:59.992788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:15:59.992788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:15:59.992788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:15:59.992788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:15:59.992788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:15:59.992788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:15:59.992788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:15:59.992788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:15:59.992788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:15:59.992788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:15:59.993857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 480/936] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Sep 12 23:15:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:15:59.996233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26dbb3-8c02-46ff-a052-c04aa5d705b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:15:59.997049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26dbb3-8c02-46ff-a052-c04aa5d705b3 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:15:59.997574 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26dbb3-8c02-46ff-a052-c04aa5d705b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:15:59.997970 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26dbb3-8c02-46ff-a052-c04aa5d705b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:15:59.998760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26dbb3-8c02-46ff-a052-c04aa5d705b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:15:59.999183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26dbb3-8c02-46ff-a052-c04aa5d705b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:15:59.999294 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990f9b7-20b9-473a-9c48-ec1cc91dcc81 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:00.000813 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26dbb3-8c02-46ff-a052-c04aa5d705b3 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:00.018645 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 457/937] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:00.019518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26dbb3-8c02-46ff-a052-c04aa5d705b3 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:00.046290 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:00.118030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990f9b7-20b9-473a-9c48-ec1cc91dcc81 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:00.135280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26dbb3-8c02-46ff-a052-c04aa5d705b3 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:00.154869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5306c72-810a-469f-b2be-13ba4efc0803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:00.155542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5306c72-810a-469f-b2be-13ba4efc0803 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:00.155851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5306c72-810a-469f-b2be-13ba4efc0803 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:00.156122 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5306c72-810a-469f-b2be-13ba4efc0803 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:00.156747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5306c72-810a-469f-b2be-13ba4efc0803 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:00.157050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5306c72-810a-469f-b2be-13ba4efc0803 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:00.158674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5306c72-810a-469f-b2be-13ba4efc0803 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:00.163600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 481/938] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Sep 12 23:15:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:00.169374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d957048-528b-49e4-bedf-73c5f7e1a590 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:00.169858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d957048-528b-49e4-bedf-73c5f7e1a590 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:00.170362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d957048-528b-49e4-bedf-73c5f7e1a590 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:00.171272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31f600-00fa-4f93-a030-9b6f82bc9286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:00.171923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31f600-00fa-4f93-a030-9b6f82bc9286 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:00.172201 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31f600-00fa-4f93-a030-9b6f82bc9286 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:00.172469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31f600-00fa-4f93-a030-9b6f82bc9286 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:00.175285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 482/939] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:00.176149 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d957048-528b-49e4-bedf-73c5f7e1a590 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:00.183640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31f600-00fa-4f93-a030-9b6f82bc9286 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:00.185530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31f600-00fa-4f93-a030-9b6f82bc9286 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-de31f600-00fa-4f93-a030-9b6f82bc9286 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:16:00.197474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:00.198371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:00.199328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:16:00.199328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:16:00.199328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:16:00.199328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:16:00.199328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:16:00.199328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:16:00.199328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:16:00.199328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:00.203610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 458/940] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:16:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:00.204696 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6e477-a1a7-40b3-8e33-6a8a1f0b2a49 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:00.204945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6e477-a1a7-40b3-8e33-6a8a1f0b2a49 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:00.205221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6e477-a1a7-40b3-8e33-6a8a1f0b2a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:00.207014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:00.223256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-e7d6e477-a1a7-40b3-8e33-6a8a1f0b2a49 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:00.224475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e7d6e477-a1a7-40b3-8e33-6a8a1f0b2a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:00.284273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d957048-528b-49e4-bedf-73c5f7e1a590 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:00.299757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 459/941] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:00.299757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6e477-a1a7-40b3-8e33-6a8a1f0b2a49 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:00.312858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:00.336597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50cbf6-d727-40a4-af6b-d8566a61d013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:00.337218 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50cbf6-d727-40a4-af6b-d8566a61d013 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:00.337497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50cbf6-d727-40a4-af6b-d8566a61d013 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:00.337771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50cbf6-d727-40a4-af6b-d8566a61d013 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:00.338513 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50cbf6-d727-40a4-af6b-d8566a61d013 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:00.338815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50cbf6-d727-40a4-af6b-d8566a61d013 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:00.340311 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50cbf6-d727-40a4-af6b-d8566a61d013 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:00.350592 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 483/942] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Sep 12 23:16:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:00.351488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50cbf6-d727-40a4-af6b-d8566a61d013 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:00.428584 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6e477-a1a7-40b3-8e33-6a8a1f0b2a49 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:00.464171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:00.478118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50cbf6-d727-40a4-af6b-d8566a61d013 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:00.482797 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b9a67-fda2-4657-b5da-b05c4f53a679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:00.483850 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b9a67-fda2-4657-b5da-b05c4f53a679 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:00.484296 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b9a67-fda2-4657-b5da-b05c4f53a679 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:00.484819 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b9a67-fda2-4657-b5da-b05c4f53a679 None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:00.486033 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b9a67-fda2-4657-b5da-b05c4f53a679 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:00.486571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b9a67-fda2-4657-b5da-b05c4f53a679 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:00.489344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b9a67-fda2-4657-b5da-b05c4f53a679 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:00.516038 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 460/943] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:16:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:00.517086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c69b4-28cd-4a76-839f-156757e7028c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:00.517857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c69b4-28cd-4a76-839f-156757e7028c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:00.518201 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b9a67-fda2-4657-b5da-b05c4f53a679 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:00.518450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c69b4-28cd-4a76-839f-156757e7028c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:00.518864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c69b4-28cd-4a76-839f-156757e7028c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:00.519775 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c69b4-28cd-4a76-839f-156757e7028c None admin] RBAC: Adding query filter params (interface=public, service_id=04cdb8323f9c40eaa0ebbe9f08bea409, region_id=RegionOne) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:00.520230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c69b4-28cd-4a76-839f-156757e7028c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:00.521740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c69b4-28cd-4a76-839f-156757e7028c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:00.530830 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 484/944] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Sep 12 23:16:00 2022] GET /identity/v3/endpoints?service_id=04cdb8323f9c40eaa0ebbe9f08bea409&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:00.538911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d87c9-6c22-4fe0-bab0-1e9903e961bf None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:00.539191 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d87c9-6c22-4fe0-bab0-1e9903e961bf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:00.539464 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d87c9-6c22-4fe0-bab0-1e9903e961bf None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:00.543480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 485/945] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:00.544098 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d87c9-6c22-4fe0-bab0-1e9903e961bf None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:00.606859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:00.637064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b9a67-fda2-4657-b5da-b05c4f53a679 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:00.652782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d87c9-6c22-4fe0-bab0-1e9903e961bf None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:00.670112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07655015-9cce-4cd6-b5fd-33cc0e6381db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:00.670758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07655015-9cce-4cd6-b5fd-33cc0e6381db None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:00.671034 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07655015-9cce-4cd6-b5fd-33cc0e6381db None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:00.671305 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07655015-9cce-4cd6-b5fd-33cc0e6381db None admin] PATH_INFO: `/v3/projects/32bbb936501846e8914acf07890c2891/users/0212eb5841a04075a0fd7ba853231267/roles/9940e26019d341f4a9af2fa399a02bc8` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:00.673470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07655015-9cce-4cd6-b5fd-33cc0e6381db None admin] RBAC: Authorizing `identity:create_grant(project_id=32bbb936501846e8914acf07890c2891, user_id=0212eb5841a04075a0fd7ba853231267, role_id=9940e26019d341f4a9af2fa399a02bc8)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:00.674975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07655015-9cce-4cd6-b5fd-33cc0e6381db None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:00.686117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226a68c-cabf-4435-8778-f279bf4799d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:00.686723 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226a68c-cabf-4435-8778-f279bf4799d3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:00.687004 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226a68c-cabf-4435-8778-f279bf4799d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:00.687280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226a68c-cabf-4435-8778-f279bf4799d3 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:00.687678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 461/946] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:16:00 2022] PUT /identity/v3/projects/32bbb936501846e8914acf07890c2891/users/0212eb5841a04075a0fd7ba853231267/roles/9940e26019d341f4a9af2fa399a02bc8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:16:00.688825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd816a7-1176-444a-9299-1e1651d0190f None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:00.689108 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd816a7-1176-444a-9299-1e1651d0190f None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:00.689380 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd816a7-1176-444a-9299-1e1651d0190f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:00.691909 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226a68c-cabf-4435-8778-f279bf4799d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:00.693381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226a68c-cabf-4435-8778-f279bf4799d3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:00.701390 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 486/947] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Sep 12 23:16:00 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:16:00.702651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226a68c-cabf-4435-8778-f279bf4799d3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:00.724170 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-bfd816a7-1176-444a-9299-1e1651d0190f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:00.726254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bfd816a7-1176-444a-9299-1e1651d0190f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:00.731592 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:00.835656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 462/948] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:00.846353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226a68c-cabf-4435-8778-f279bf4799d3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:00.897756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c6802-a676-42b7-8e8f-3b30dd41e004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:00.898790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c6802-a676-42b7-8e8f-3b30dd41e004 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:00.899072 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c6802-a676-42b7-8e8f-3b30dd41e004 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:00.899353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c6802-a676-42b7-8e8f-3b30dd41e004 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:00.900115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c6802-a676-42b7-8e8f-3b30dd41e004 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:00.901649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c6802-a676-42b7-8e8f-3b30dd41e004 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5a6c6802-a676-42b7-8e8f-3b30dd41e004 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:00.917207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:00.918237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:00.919064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 487/949] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:16:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:00.922180 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd816a7-1176-444a-9299-1e1651d0190f None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:00.989778 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:01.030936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd816a7-1176-444a-9299-1e1651d0190f None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:01.070560 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eccdc2-fbd8-4b21-a5f9-fe2828466e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:01.071328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eccdc2-fbd8-4b21-a5f9-fe2828466e70 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:01.071545 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eccdc2-fbd8-4b21-a5f9-fe2828466e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:01.071848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eccdc2-fbd8-4b21-a5f9-fe2828466e70 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:01.072479 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eccdc2-fbd8-4b21-a5f9-fe2828466e70 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:01.072840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eccdc2-fbd8-4b21-a5f9-fe2828466e70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:01.074223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eccdc2-fbd8-4b21-a5f9-fe2828466e70 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:01.079481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 463/950] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Sep 12 23:16:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:01.085651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c6802-a676-42b7-8e8f-3b30dd41e004 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:01.117256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eccdc2-fbd8-4b21-a5f9-fe2828466e70 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:01.184383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:01.205875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c6802-a676-42b7-8e8f-3b30dd41e004 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:01.249103 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bcd2e-e95b-43ab-ad06-99874d60e497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:01.249639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bcd2e-e95b-43ab-ad06-99874d60e497 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:01.249920 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bcd2e-e95b-43ab-ad06-99874d60e497 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:01.250195 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bcd2e-e95b-43ab-ad06-99874d60e497 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:01.250822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33bcd2e-e95b-43ab-ad06-99874d60e497 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:01.251147 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33bcd2e-e95b-43ab-ad06-99874d60e497 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:01.252724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33bcd2e-e95b-43ab-ad06-99874d60e497 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:01.261216 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eccdc2-fbd8-4b21-a5f9-fe2828466e70 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:01.263658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 488/951] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Sep 12 23:16:01 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:01.301576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:01.312576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1982338-96cc-4680-be40-6cd65edfe239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:01.313671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1982338-96cc-4680-be40-6cd65edfe239 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:01.314159 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1982338-96cc-4680-be40-6cd65edfe239 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:01.314680 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1982338-96cc-4680-be40-6cd65edfe239 None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:01.315560 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1982338-96cc-4680-be40-6cd65edfe239 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:01.317341 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1982338-96cc-4680-be40-6cd65edfe239 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:01.321000 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: WARNING keystone.server.flask.common [None req-c1982338-96cc-4680-be40-6cd65edfe239 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 12 23:16:01.343235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.sql.core [None req-c1982338-96cc-4680-be40-6cd65edfe239 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 23:16:01.343235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 12 23:16:01.343235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [parameters: {'id': '552e1b4ff76643429568f9087f845881', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 12 23:16:01.343235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81416) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 23:16:01.352810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-c1982338-96cc-4680-be40-6cd65edfe239 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 552e1b4ff76643429568f9087f845881.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 552e1b4ff76643429568f9087f845881. Sep 12 23:16:01.352810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:01.352810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 23:16:01.352810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 23:16:01.352810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 23:16:01.352810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 23:16:01.352810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 23:16:01.352810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 23:16:01.352810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 23:16:01.352810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application conn.query(q) Sep 12 23:16:01.352810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 23:16:01.352810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 23:16:01.352810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application result.read() Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 12 23:16:01.353736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application next(self.gen) Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application yield resource Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application next(self.gen) Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise self.value Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application session.commit() Sep 12 23:16:01.354724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.session.flush() Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application compat.raise_( Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 23:16:01.355661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 23:16:01.356593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application conn.query(q) Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application result.read() Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 23:16:01.357403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [parameters: {'id': '552e1b4ff76643429568f9087f845881', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:01.359633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:01.361542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:01.363889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:01.363889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 12 23:16:01.363889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 12 23:16:01.363889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:01.363889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:01.363889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 12 23:16:01.363889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 12 23:16:01.363889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 552e1b4ff76643429568f9087f845881. Sep 12 23:16:01.363889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:01.371787 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 464/952] 10.222.0.27 () {60 vars in 1140 bytes} [Mon Sep 12 23:16:01 2022] POST /identity/v3/projects => generated 262 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:01.374350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bcd2e-e95b-43ab-ad06-99874d60e497 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:01.487800 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bcd2e-e95b-43ab-ad06-99874d60e497 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:01.528624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd5703-30ee-43d6-905e-9f5164de8f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:01.529179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd5703-30ee-43d6-905e-9f5164de8f2f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:01.529458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd5703-30ee-43d6-905e-9f5164de8f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:01.529728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd5703-30ee-43d6-905e-9f5164de8f2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:01.537437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd5703-30ee-43d6-905e-9f5164de8f2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:01.538892 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd5703-30ee-43d6-905e-9f5164de8f2f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8fcd5703-30ee-43d6-905e-9f5164de8f2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:16:01.551234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:01.552266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:01.553323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:16:01.553323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:16:01.553323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:16:01.553323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:16:01.553323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:16:01.553323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:16:01.553323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:16:01.553323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:01.553323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 489/953] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:16:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:01.560360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1982338-96cc-4680-be40-6cd65edfe239 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:01.665575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1982338-96cc-4680-be40-6cd65edfe239 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:01.703520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb67cf-3dad-4ba1-83d4-d284b23b74e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:01.704178 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb67cf-3dad-4ba1-83d4-d284b23b74e7 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:01.704454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb67cf-3dad-4ba1-83d4-d284b23b74e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:01.704723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb67cf-3dad-4ba1-83d4-d284b23b74e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:01.705407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb67cf-3dad-4ba1-83d4-d284b23b74e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:01.705710 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb67cf-3dad-4ba1-83d4-d284b23b74e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:01.707409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb67cf-3dad-4ba1-83d4-d284b23b74e7 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:01.734121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 465/954] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Sep 12 23:16:01 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:02.977219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0762025-ba71-4d1c-bd74-0f1ce4117e8e None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:02.978222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0762025-ba71-4d1c-bd74-0f1ce4117e8e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:02.978757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0762025-ba71-4d1c-bd74-0f1ce4117e8e None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:02.980660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 490/955] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:02.986552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718db0b-3c26-4d7e-ad0e-ce08acdbab8c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:02.987395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718db0b-3c26-4d7e-ad0e-ce08acdbab8c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:02.987932 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718db0b-3c26-4d7e-ad0e-ce08acdbab8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:02.992078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f8451-689b-49e2-b73f-d00a057dab9b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:02.992366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f8451-689b-49e2-b73f-d00a057dab9b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:02.992645 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f8451-689b-49e2-b73f-d00a057dab9b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:02.994199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 491/956] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:16:02.999343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60856fd-2836-49e9-8ae8-ede5a3984ffe None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:02.999655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60856fd-2836-49e9-8ae8-ede5a3984ffe None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:02.999931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60856fd-2836-49e9-8ae8-ede5a3984ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:03.010936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-f718db0b-3c26-4d7e-ad0e-ce08acdbab8c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:03.012994 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-f718db0b-3c26-4d7e-ad0e-ce08acdbab8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:03.018413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d60856fd-2836-49e9-8ae8-ede5a3984ffe None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:03.019868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d60856fd-2836-49e9-8ae8-ede5a3984ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:03.022334 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:03.055060 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:03.095140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 466/957] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:03.111492 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 492/958] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:03.373999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718db0b-3c26-4d7e-ad0e-ce08acdbab8c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:03.378650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60856fd-2836-49e9-8ae8-ede5a3984ffe None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:03.507940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718db0b-3c26-4d7e-ad0e-ce08acdbab8c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:03.511214 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60856fd-2836-49e9-8ae8-ede5a3984ffe None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:03.546103 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561a244-d39b-4aca-b866-2a50e120aa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:03.546730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561a244-d39b-4aca-b866-2a50e120aa99 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:03.547014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561a244-d39b-4aca-b866-2a50e120aa99 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:03.547287 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561a244-d39b-4aca-b866-2a50e120aa99 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:03.547911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561a244-d39b-4aca-b866-2a50e120aa99 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:03.549263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561a244-d39b-4aca-b866-2a50e120aa99 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:03.551076 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627239b-3d78-447a-964d-c3dcfd06939c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:03.552082 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627239b-3d78-447a-964d-c3dcfd06939c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:03.552365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627239b-3d78-447a-964d-c3dcfd06939c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:03.552650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627239b-3d78-447a-964d-c3dcfd06939c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8561a244-d39b-4aca-b866-2a50e120aa99 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:03.558578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:03.559962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:03.560914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 493/959] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Sep 12 23:16:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:03.561731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226ea00-8880-4d4b-b0b2-582bd8a5aeb6 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:03.561731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226ea00-8880-4d4b-b0b2-582bd8a5aeb6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:03.562274 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226ea00-8880-4d4b-b0b2-582bd8a5aeb6 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:03.563288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 494/960] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:03.565797 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226ea00-8880-4d4b-b0b2-582bd8a5aeb6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:03.569348 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:03.572140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627239b-3d78-447a-964d-c3dcfd06939c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-7627239b-3d78-447a-964d-c3dcfd06939c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:16:03.577744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:03.578728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:03.579983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:03.579983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:03.579983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:03.579983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:03.579983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:16:03.579983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:16:03.579983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:16:03.579983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:16:03.579983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:16:03.579983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:16:03.579983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:16:03.579983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:03.584785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 467/961] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Sep 12 23:16:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:03.586304 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4efbec-c387-452e-90a9-a6e4ce0231b9 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:03.588538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4efbec-c387-452e-90a9-a6e4ce0231b9 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:03.588854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4efbec-c387-452e-90a9-a6e4ce0231b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:03.608664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-9a4efbec-c387-452e-90a9-a6e4ce0231b9 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:03.609954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9a4efbec-c387-452e-90a9-a6e4ce0231b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:03.636348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:03.679912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 468/962] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:03.682429 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226ea00-8880-4d4b-b0b2-582bd8a5aeb6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:03.688527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4efbec-c387-452e-90a9-a6e4ce0231b9 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:03.719514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e0513-460d-4322-9f91-ad10d5f24a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:03.720163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e0513-460d-4322-9f91-ad10d5f24a32 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:03.720449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e0513-460d-4322-9f91-ad10d5f24a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:03.720726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e0513-460d-4322-9f91-ad10d5f24a32 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:03.721354 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e0513-460d-4322-9f91-ad10d5f24a32 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:03.721658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e0513-460d-4322-9f91-ad10d5f24a32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:03.723064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e0513-460d-4322-9f91-ad10d5f24a32 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:03.733227 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 495/963] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Sep 12 23:16:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:03.735190 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e0513-460d-4322-9f91-ad10d5f24a32 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:03.811118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:03.819119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4efbec-c387-452e-90a9-a6e4ce0231b9 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:03.857955 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ae0ae-b7f5-4244-a63c-7ac71ab912e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:03.858626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ae0ae-b7f5-4244-a63c-7ac71ab912e1 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:03.858973 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ae0ae-b7f5-4244-a63c-7ac71ab912e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:03.859175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ae0ae-b7f5-4244-a63c-7ac71ab912e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:03.859941 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ae0ae-b7f5-4244-a63c-7ac71ab912e1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:03.860244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ae0ae-b7f5-4244-a63c-7ac71ab912e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:03.861759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ae0ae-b7f5-4244-a63c-7ac71ab912e1 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:03.865374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e0513-460d-4322-9f91-ad10d5f24a32 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:03.875702 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 469/964] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Sep 12 23:16:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:03.877418 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ae0ae-b7f5-4244-a63c-7ac71ab912e1 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:03.910879 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029589b1-324d-4d8f-b5c5-8137863e7a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:03.911429 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029589b1-324d-4d8f-b5c5-8137863e7a41 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:03.911843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029589b1-324d-4d8f-b5c5-8137863e7a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:03.912126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029589b1-324d-4d8f-b5c5-8137863e7a41 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:03.913304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029589b1-324d-4d8f-b5c5-8137863e7a41 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:03.913646 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029589b1-324d-4d8f-b5c5-8137863e7a41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:03.914736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:03.914962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029589b1-324d-4d8f-b5c5-8137863e7a41 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:03.925880 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 496/965] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Sep 12 23:16:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:03.927303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029589b1-324d-4d8f-b5c5-8137863e7a41 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:04.003077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ae0ae-b7f5-4244-a63c-7ac71ab912e1 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:04.035133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029589b1-324d-4d8f-b5c5-8137863e7a41 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:04.036739 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2da156-6290-4db4-aa9c-bb69f71db90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:04.037941 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2da156-6290-4db4-aa9c-bb69f71db90c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:04.038235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2da156-6290-4db4-aa9c-bb69f71db90c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:04.038490 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2da156-6290-4db4-aa9c-bb69f71db90c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:04.048333 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2da156-6290-4db4-aa9c-bb69f71db90c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:04.049981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2da156-6290-4db4-aa9c-bb69f71db90c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-2c2da156-6290-4db4-aa9c-bb69f71db90c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:16:04.064236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:04.065236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:16:04.066150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:16:04.066150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:04.066150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:04.066150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 470/966] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:04.066150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2da156-6290-4db4-aa9c-bb69f71db90c None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:04.070355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:04.076736 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc066953-8d84-4808-833f-87e5f134167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:04.077323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc066953-8d84-4808-833f-87e5f134167d None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:04.077602 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc066953-8d84-4808-833f-87e5f134167d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:04.077871 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc066953-8d84-4808-833f-87e5f134167d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:04.078529 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc066953-8d84-4808-833f-87e5f134167d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:04.079923 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc066953-8d84-4808-833f-87e5f134167d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:04.099519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 497/967] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:16:03 2022] POST /identity/v3/endpoints => generated 349 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:16:04.100812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9588c0-265f-4fc1-b436-cf07647450c7 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:04.101097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9588c0-265f-4fc1-b436-cf07647450c7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:04.101372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9588c0-265f-4fc1-b436-cf07647450c7 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:04.102565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 498/968] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:04.103265 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9588c0-265f-4fc1-b436-cf07647450c7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:04.167720 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:04.193563 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2da156-6290-4db4-aa9c-bb69f71db90c None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:04.232415 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9588c0-265f-4fc1-b436-cf07647450c7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:04.232627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecbebc-5641-40c8-ae2d-09c12aafebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:04.233181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecbebc-5641-40c8-ae2d-09c12aafebd0 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:04.233460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecbebc-5641-40c8-ae2d-09c12aafebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:04.233734 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecbebc-5641-40c8-ae2d-09c12aafebd0 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:04.234516 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecbebc-5641-40c8-ae2d-09c12aafebd0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:04.236070 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecbebc-5641-40c8-ae2d-09c12aafebd0 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:04.256006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 471/969] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Sep 12 23:16:04 2022] POST /identity/v3/services => generated 242 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:16:04.257608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38d1b3-df01-4fde-9b13-163412b5180d None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:04.257899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38d1b3-df01-4fde-9b13-163412b5180d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:04.258333 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38d1b3-df01-4fde-9b13-163412b5180d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:04.281410 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1420c-f518-4436-b78b-9f32a79ec69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:04.282731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1420c-f518-4436-b78b-9f32a79ec69f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:04.283302 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1420c-f518-4436-b78b-9f32a79ec69f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:04.283799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1420c-f518-4436-b78b-9f32a79ec69f None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:04.284136 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-1b38d1b3-df01-4fde-9b13-163412b5180d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:04.285023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1420c-f518-4436-b78b-9f32a79ec69f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:04.285406 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1b38d1b3-df01-4fde-9b13-163412b5180d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:04.285811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1420c-f518-4436-b78b-9f32a79ec69f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:04.288421 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1420c-f518-4436-b78b-9f32a79ec69f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:04.307123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 499/970] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:04.315177 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1420c-f518-4436-b78b-9f32a79ec69f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:04.369934 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:04.371112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 472/971] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:04.438176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1420c-f518-4436-b78b-9f32a79ec69f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:04.473013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32d4fd-3263-45c4-a79e-359101c7eef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:04.473646 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32d4fd-3263-45c4-a79e-359101c7eef4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:04.473925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32d4fd-3263-45c4-a79e-359101c7eef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:04.474200 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32d4fd-3263-45c4-a79e-359101c7eef4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:04.482028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32d4fd-3263-45c4-a79e-359101c7eef4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:04.483464 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32d4fd-3263-45c4-a79e-359101c7eef4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ab32d4fd-3263-45c4-a79e-359101c7eef4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:16:04.497467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:04.498561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:04.499544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:04.499544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:04.499544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:16:04.499544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:16:04.499544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:16:04.499544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:16:04.499544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:16:04.499544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:16:04.499544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:16:04.499544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:04.499544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 500/972] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Sep 12 23:16:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:04.507034 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38d1b3-df01-4fde-9b13-163412b5180d None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:04.582107 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:04.618481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38d1b3-df01-4fde-9b13-163412b5180d None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:04.653411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fd308-e478-4037-87a5-db9accc28ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:04.654027 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fd308-e478-4037-87a5-db9accc28ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:04.654305 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fd308-e478-4037-87a5-db9accc28ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:04.654574 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fd308-e478-4037-87a5-db9accc28ba9 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:04.655300 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fd308-e478-4037-87a5-db9accc28ba9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:04.658404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fd308-e478-4037-87a5-db9accc28ba9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:04.658404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fd308-e478-4037-87a5-db9accc28ba9 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:04.675413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 473/973] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Sep 12 23:16:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:04.681288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32d4fd-3263-45c4-a79e-359101c7eef4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:04.689228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fd308-e478-4037-87a5-db9accc28ba9 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:04.795908 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:04.802938 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32d4fd-3263-45c4-a79e-359101c7eef4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:04.807437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fd308-e478-4037-87a5-db9accc28ba9 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:04.839802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a479295-ada3-4736-ba5c-6115603ae0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:04.840401 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a479295-ada3-4736-ba5c-6115603ae0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:04.840652 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a479295-ada3-4736-ba5c-6115603ae0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:04.840928 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a479295-ada3-4736-ba5c-6115603ae0a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:04.847986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:04.853992 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b127e3f-7d09-4b3b-bc51-c8e6ef7ec023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:04.855167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b127e3f-7d09-4b3b-bc51-c8e6ef7ec023 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:04.855728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b127e3f-7d09-4b3b-bc51-c8e6ef7ec023 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:04.856468 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b127e3f-7d09-4b3b-bc51-c8e6ef7ec023 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:04.857824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b127e3f-7d09-4b3b-bc51-c8e6ef7ec023 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:04.860030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b127e3f-7d09-4b3b-bc51-c8e6ef7ec023 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:04.877966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.sql.core [None req-8b127e3f-7d09-4b3b-bc51-c8e6ef7ec023 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 23:16:04.877966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 12 23:16:04.877966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [parameters: {'user_id': '5a4606ec067041f6b12cc1515c948284', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 12 23:16:04.877966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81416) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-8b127e3f-7d09-4b3b-bc51-c8e6ef7ec023 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application conn.query(q) Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application result.read() Sep 12 23:16:04.885654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 23:16:04.886569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application next(self.gen) Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application yield resource Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application next(self.gen) Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise self.value Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application session.commit() Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 23:16:04.887555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.session.flush() Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application compat.raise_( Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 23:16:04.889017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 23:16:04.889903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application conn.query(q) Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application result.read() Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 23:16:04.891815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [parameters: {'user_id': '5a4606ec067041f6b12cc1515c948284', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:04.894047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 12 23:16:04.895975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 12 23:16:04.899849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 12 23:16:04.899849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 12 23:16:04.899849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 23:16:04.899849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 23:16:04.899849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 12 23:16:04.899849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:04.899849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 474/974] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Sep 12 23:16:04 2022] POST /identity/v3/users => generated 160 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:04.899849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b127e3f-7d09-4b3b-bc51-c8e6ef7ec023 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:04.940869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a479295-ada3-4736-ba5c-6115603ae0a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:04.942726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a479295-ada3-4736-ba5c-6115603ae0a1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:05.010526 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b127e3f-7d09-4b3b-bc51-c8e6ef7ec023 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:05.052344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff46609-1d91-4eec-83f6-8c76169bdb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:05.053077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff46609-1d91-4eec-83f6-8c76169bdb57 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:05.053359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff46609-1d91-4eec-83f6-8c76169bdb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:05.053631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff46609-1d91-4eec-83f6-8c76169bdb57 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:05.061583 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff46609-1d91-4eec-83f6-8c76169bdb57 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:05.063403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff46609-1d91-4eec-83f6-8c76169bdb57 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:05.067751 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:05.076090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 501/975] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:16:04 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:16:05.080487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a479295-ada3-4736-ba5c-6115603ae0a1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-6ff46609-1d91-4eec-83f6-8c76169bdb57 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:16:05.084745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:05.085709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:05.087166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:05.087166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:05.087166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:16:05.087166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:16:05.087166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:16:05.087166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:16:05.087166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:16:05.087166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:16:05.087166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:16:05.087166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:05.090420 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 475/976] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:16:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:05.091300 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff46609-1d91-4eec-83f6-8c76169bdb57 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:05.152141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:05.207807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a479295-ada3-4736-ba5c-6115603ae0a1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:05.230578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff46609-1d91-4eec-83f6-8c76169bdb57 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:05.244990 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ece5d-d9ad-429d-9982-687e6dd48fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:05.245625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ece5d-d9ad-429d-9982-687e6dd48fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:05.245906 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ece5d-d9ad-429d-9982-687e6dd48fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:05.246180 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ece5d-d9ad-429d-9982-687e6dd48fc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:05.257022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ece5d-d9ad-429d-9982-687e6dd48fc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:05.258459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ece5d-d9ad-429d-9982-687e6dd48fc5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d94ece5d-d9ad-429d-9982-687e6dd48fc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:16:05.273624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:05.274590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:16:05.275501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:16:05.275501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:05.275501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:05.276797 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e020fcd-e6aa-40ec-99a8-df243e96d23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:05.277739 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e020fcd-e6aa-40ec-99a8-df243e96d23b None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:05.278236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e020fcd-e6aa-40ec-99a8-df243e96d23b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:05.278704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e020fcd-e6aa-40ec-99a8-df243e96d23b None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:05.279607 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 502/977] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:05.280092 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e020fcd-e6aa-40ec-99a8-df243e96d23b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:05.280607 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e020fcd-e6aa-40ec-99a8-df243e96d23b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:05.281061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ece5d-d9ad-429d-9982-687e6dd48fc5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:05.282853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e020fcd-e6aa-40ec-99a8-df243e96d23b None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:05.340112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 476/978] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Sep 12 23:16:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:05.340503 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:05.399342 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ece5d-d9ad-429d-9982-687e6dd48fc5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:05.438168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb32674-34e1-4e25-8ca9-8426c38dac40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:05.439165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb32674-34e1-4e25-8ca9-8426c38dac40 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:05.439676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb32674-34e1-4e25-8ca9-8426c38dac40 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:05.440153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb32674-34e1-4e25-8ca9-8426c38dac40 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:05.441033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb32674-34e1-4e25-8ca9-8426c38dac40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:05.441549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb32674-34e1-4e25-8ca9-8426c38dac40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:05.443142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb32674-34e1-4e25-8ca9-8426c38dac40 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:05.457222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 503/979] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:05.466035 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e020fcd-e6aa-40ec-99a8-df243e96d23b None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:05.573499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e020fcd-e6aa-40ec-99a8-df243e96d23b None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:05.575964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:05.611743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cd399-63dc-4861-8f82-a6a2f75e9a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:05.612388 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cd399-63dc-4861-8f82-a6a2f75e9a77 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:05.612669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cd399-63dc-4861-8f82-a6a2f75e9a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:05.613010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cd399-63dc-4861-8f82-a6a2f75e9a77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:05.622459 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4cd399-63dc-4861-8f82-a6a2f75e9a77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:05.624899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4cd399-63dc-4861-8f82-a6a2f75e9a77 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-ae4cd399-63dc-4861-8f82-a6a2f75e9a77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:16:05.642048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:05.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:05.644346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:16:05.644346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:16:05.644346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:16:05.644346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:16:05.644346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:16:05.644346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:16:05.644346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:16:05.644346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:05.644346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 477/980] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:16:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:05.649957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb32674-34e1-4e25-8ca9-8426c38dac40 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:05.753715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb32674-34e1-4e25-8ca9-8426c38dac40 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:05.764296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:05.789826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762e44e-5b51-4814-ae00-6650ddfe66c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:05.790439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762e44e-5b51-4814-ae00-6650ddfe66c7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:05.790722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762e44e-5b51-4814-ae00-6650ddfe66c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:05.791000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762e44e-5b51-4814-ae00-6650ddfe66c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:05.791718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762e44e-5b51-4814-ae00-6650ddfe66c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:05.792030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762e44e-5b51-4814-ae00-6650ddfe66c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:05.793450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762e44e-5b51-4814-ae00-6650ddfe66c7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:05.805080 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 504/981] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:16:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:05.810898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cd399-63dc-4861-8f82-a6a2f75e9a77 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:05.910673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cd399-63dc-4861-8f82-a6a2f75e9a77 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:05.951937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2aa07-e5d0-44cf-8e71-19cb44c50472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:05.952548 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2aa07-e5d0-44cf-8e71-19cb44c50472 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:05.952829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2aa07-e5d0-44cf-8e71-19cb44c50472 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:05.953103 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2aa07-e5d0-44cf-8e71-19cb44c50472 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:05.953834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2aa07-e5d0-44cf-8e71-19cb44c50472 None admin] RBAC: Adding query filter params (role.id=9940e26019d341f4a9af2fa399a02bc8, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=0212eb5841a04075a0fd7ba853231267) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:05.954142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2aa07-e5d0-44cf-8e71-19cb44c50472 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:05.955654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2aa07-e5d0-44cf-8e71-19cb44c50472 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:05.967622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 478/982] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:16:05 2022] GET /identity/v3/role_assignments?user.id=0212eb5841a04075a0fd7ba853231267&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=9940e26019d341f4a9af2fa399a02bc8 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:06.388666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f75790-fff4-4af7-818c-1e0961e33ca2 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:06.391640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f75790-fff4-4af7-818c-1e0961e33ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:06.391640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f75790-fff4-4af7-818c-1e0961e33ca2 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:06.391640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 505/983] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:06.396994 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9df1ea-6757-4be3-bd02-51aa7824df8c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:06.397487 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9df1ea-6757-4be3-bd02-51aa7824df8c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:06.397676 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9df1ea-6757-4be3-bd02-51aa7824df8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:06.415988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-1c9df1ea-6757-4be3-bd02-51aa7824df8c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:06.417504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1c9df1ea-6757-4be3-bd02-51aa7824df8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:06.440772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:06.465313 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9ce4-d118-4f78-b9c5-a99192a938a3 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:06.465605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9ce4-d118-4f78-b9c5-a99192a938a3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:06.465889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9ce4-d118-4f78-b9c5-a99192a938a3 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:06.467363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 506/984] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:16:06.472472 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82f4a6-e138-43d9-b624-5587ae8550d2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:06.472757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82f4a6-e138-43d9-b624-5587ae8550d2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:06.473040 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82f4a6-e138-43d9-b624-5587ae8550d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:06.489717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 479/985] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:06.491993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ec82f4a6-e138-43d9-b624-5587ae8550d2 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:06.493182 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ec82f4a6-e138-43d9-b624-5587ae8550d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:06.562433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 507/986] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:16:06.756793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9df1ea-6757-4be3-bd02-51aa7824df8c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:06.827728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82f4a6-e138-43d9-b624-5587ae8550d2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:06.855930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9df1ea-6757-4be3-bd02-51aa7824df8c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:06.894344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468024ca-70c0-45ed-bec1-71f09f21fb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:06.895245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468024ca-70c0-45ed-bec1-71f09f21fb5f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:06.895726 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468024ca-70c0-45ed-bec1-71f09f21fb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:06.896159 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468024ca-70c0-45ed-bec1-71f09f21fb5f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:06.897403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468024ca-70c0-45ed-bec1-71f09f21fb5f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:06.898941 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468024ca-70c0-45ed-bec1-71f09f21fb5f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-468024ca-70c0-45ed-bec1-71f09f21fb5f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:06.909125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:06.910225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:06.911129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 480/987] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Sep 12 23:16:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:06.916602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468024ca-70c0-45ed-bec1-71f09f21fb5f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:06.938309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:06.942337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82f4a6-e138-43d9-b624-5587ae8550d2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:06.945736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:06.976933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf9398-7b65-45df-b196-16a57ec31438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:06.977553 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf9398-7b65-45df-b196-16a57ec31438 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:06.977824 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf9398-7b65-45df-b196-16a57ec31438 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:06.978103 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf9398-7b65-45df-b196-16a57ec31438 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:06.978672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf9398-7b65-45df-b196-16a57ec31438 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:06.980051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf9398-7b65-45df-b196-16a57ec31438 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4cbf9398-7b65-45df-b196-16a57ec31438 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:06.988588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:06.989739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:06.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 508/988] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:16:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:06.995939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf9398-7b65-45df-b196-16a57ec31438 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:07.035383 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468024ca-70c0-45ed-bec1-71f09f21fb5f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:07.068484 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f37cc-e3f2-4306-97ae-76fd5c950dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:07.069089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f37cc-e3f2-4306-97ae-76fd5c950dfa None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:07.069359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f37cc-e3f2-4306-97ae-76fd5c950dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:07.069634 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f37cc-e3f2-4306-97ae-76fd5c950dfa None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:07.070254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f37cc-e3f2-4306-97ae-76fd5c950dfa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:07.070555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f37cc-e3f2-4306-97ae-76fd5c950dfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:07.071943 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f37cc-e3f2-4306-97ae-76fd5c950dfa None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:07.077137 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 481/989] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Sep 12 23:16:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:07.082414 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f37cc-e3f2-4306-97ae-76fd5c950dfa None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:07.106106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf9398-7b65-45df-b196-16a57ec31438 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:07.142537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e2ed8-922b-4ecb-9850-aa7e824232f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:07.143143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2ed8-922b-4ecb-9850-aa7e824232f4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:07.143416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2ed8-922b-4ecb-9850-aa7e824232f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:07.143718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2ed8-922b-4ecb-9850-aa7e824232f4 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:07.144334 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e2ed8-922b-4ecb-9850-aa7e824232f4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:07.144632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e2ed8-922b-4ecb-9850-aa7e824232f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:07.145972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e2ed8-922b-4ecb-9850-aa7e824232f4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:07.151893 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 509/990] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Sep 12 23:16:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:07.157577 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e2ed8-922b-4ecb-9850-aa7e824232f4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:07.178786 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:07.188796 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:07.204320 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f37cc-e3f2-4306-97ae-76fd5c950dfa None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:07.241803 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43472e60-5ca3-4022-9437-273b99adce7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:07.242419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43472e60-5ca3-4022-9437-273b99adce7e None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:07.242808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43472e60-5ca3-4022-9437-273b99adce7e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:07.243084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43472e60-5ca3-4022-9437-273b99adce7e None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:07.243734 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43472e60-5ca3-4022-9437-273b99adce7e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:07.244036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43472e60-5ca3-4022-9437-273b99adce7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:07.245530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43472e60-5ca3-4022-9437-273b99adce7e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:07.250826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 482/991] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Sep 12 23:16:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:07.256366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43472e60-5ca3-4022-9437-273b99adce7e None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:07.266871 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e2ed8-922b-4ecb-9850-aa7e824232f4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:07.305617 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175057ef-56eb-4cd4-9b51-a3a94d1f0ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:07.306530 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175057ef-56eb-4cd4-9b51-a3a94d1f0ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:07.306954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175057ef-56eb-4cd4-9b51-a3a94d1f0ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:07.307350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175057ef-56eb-4cd4-9b51-a3a94d1f0ae9 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:07.308020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175057ef-56eb-4cd4-9b51-a3a94d1f0ae9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:07.308325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175057ef-56eb-4cd4-9b51-a3a94d1f0ae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:07.309688 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175057ef-56eb-4cd4-9b51-a3a94d1f0ae9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:07.314928 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 510/992] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Sep 12 23:16:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:07.321569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175057ef-56eb-4cd4-9b51-a3a94d1f0ae9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:07.365626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43472e60-5ca3-4022-9437-273b99adce7e None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:07.401406 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8e95d-4558-49c1-9ccc-f769dbcfba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:07.402127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8e95d-4558-49c1-9ccc-f769dbcfba99 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:07.402417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8e95d-4558-49c1-9ccc-f769dbcfba99 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:07.402674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8e95d-4558-49c1-9ccc-f769dbcfba99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:07.403368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8e95d-4558-49c1-9ccc-f769dbcfba99 None admin] RBAC: Adding query filter params (interface=admin, service_id=04cdb8323f9c40eaa0ebbe9f08bea409, region_id=RegionOne) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:07.403704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8e95d-4558-49c1-9ccc-f769dbcfba99 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:07.405202 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8e95d-4558-49c1-9ccc-f769dbcfba99 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:07.415499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:07.416391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 483/993] 10.222.0.27 () {56 vars in 1249 bytes} [Mon Sep 12 23:16:07 2022] GET /identity/v3/endpoints?service_id=04cdb8323f9c40eaa0ebbe9f08bea409&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:07.417213 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8e95d-4558-49c1-9ccc-f769dbcfba99 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:07.425036 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175057ef-56eb-4cd4-9b51-a3a94d1f0ae9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:07.433653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:07.461965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38417661-ee09-4635-b77c-94a64a31c78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:07.462567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38417661-ee09-4635-b77c-94a64a31c78a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:07.462896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38417661-ee09-4635-b77c-94a64a31c78a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:07.463170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38417661-ee09-4635-b77c-94a64a31c78a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:07.464005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38417661-ee09-4635-b77c-94a64a31c78a None admin] RBAC: Adding query filter params (interface=public, service_id=82b9dff595f743d7934df9191603999c, region_id=RegionOne) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:07.464313 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38417661-ee09-4635-b77c-94a64a31c78a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:07.465770 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38417661-ee09-4635-b77c-94a64a31c78a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:07.471141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 511/994] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Sep 12 23:16:07 2022] GET /identity/v3/endpoints?service_id=82b9dff595f743d7934df9191603999c&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:07.477226 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38417661-ee09-4635-b77c-94a64a31c78a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:07.516808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8e95d-4558-49c1-9ccc-f769dbcfba99 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:07.549849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd896c-58b9-44a3-a1d7-f8225b7c30cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:07.550541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd896c-58b9-44a3-a1d7-f8225b7c30cb None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:07.550894 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd896c-58b9-44a3-a1d7-f8225b7c30cb None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:07.551241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd896c-58b9-44a3-a1d7-f8225b7c30cb None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:07.551940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd896c-58b9-44a3-a1d7-f8225b7c30cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:07.553462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd896c-58b9-44a3-a1d7-f8225b7c30cb None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:07.560014 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 484/995] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Sep 12 23:16:07 2022] GET /identity/v3/services => generated 1727 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:16:07.588505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38417661-ee09-4635-b77c-94a64a31c78a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:07.626345 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471041e8-230c-4c1f-a42e-27c866ece9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:07.627055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471041e8-230c-4c1f-a42e-27c866ece9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:07.627401 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471041e8-230c-4c1f-a42e-27c866ece9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:07.627769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471041e8-230c-4c1f-a42e-27c866ece9a4 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:07.628452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471041e8-230c-4c1f-a42e-27c866ece9a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:07.629893 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471041e8-230c-4c1f-a42e-27c866ece9a4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:07.643623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 512/996] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Sep 12 23:16:07 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:16:08.165068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11076da8-5156-47d8-862f-3c9ffcbb62b5 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:08.165370 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11076da8-5156-47d8-862f-3c9ffcbb62b5 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:08.165617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11076da8-5156-47d8-862f-3c9ffcbb62b5 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:08.166992 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 485/997] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:08.172096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fdd24-145d-4db9-a33c-e3ea6c3195fe None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:08.172327 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fdd24-145d-4db9-a33c-e3ea6c3195fe None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:08.172596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fdd24-145d-4db9-a33c-e3ea6c3195fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:08.190808 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ee0fdd24-145d-4db9-a33c-e3ea6c3195fe None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:08.192016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ee0fdd24-145d-4db9-a33c-e3ea6c3195fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:08.206592 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:08.260959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 513/998] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:08.526715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11076da8-5156-47d8-862f-3c9ffcbb62b5 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:08.607121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:08.637276 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11076da8-5156-47d8-862f-3c9ffcbb62b5 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:08.669552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7b55a-c4db-41fb-90ee-255619e4c647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:08.670175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7b55a-c4db-41fb-90ee-255619e4c647 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:08.670452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7b55a-c4db-41fb-90ee-255619e4c647 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:08.670723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7b55a-c4db-41fb-90ee-255619e4c647 None admin] PATH_INFO: `/v3/services/image` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:08.671295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7b55a-c4db-41fb-90ee-255619e4c647 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:08.672678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7b55a-c4db-41fb-90ee-255619e4c647 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-8dc7b55a-c4db-41fb-90ee-255619e4c647 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:08.681806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:08.682777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:08.683638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:08.683638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:08.683638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:08.683638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:08.683638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:08.683638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:08.683638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:08.683638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:08.683638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:08.683638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:08.683638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:16:08.683638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:08.683638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:16:08.683638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:08.687888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 486/999] 10.222.0.27 () {56 vars in 1105 bytes} [Mon Sep 12 23:16:08 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:08.688725 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0fdd24-145d-4db9-a33c-e3ea6c3195fe None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:08.787689 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0fdd24-145d-4db9-a33c-e3ea6c3195fe None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:08.821478 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beb250-d464-4240-a0b2-dce1063febd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:08.822091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beb250-d464-4240-a0b2-dce1063febd2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:08.822371 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beb250-d464-4240-a0b2-dce1063febd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:08.822647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beb250-d464-4240-a0b2-dce1063febd2 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:08.823277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76beb250-d464-4240-a0b2-dce1063febd2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:08.823643 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76beb250-d464-4240-a0b2-dce1063febd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:08.824966 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76beb250-d464-4240-a0b2-dce1063febd2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:08.829849 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 514/1000] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:08.839772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7b55a-c4db-41fb-90ee-255619e4c647 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:08.955072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7b55a-c4db-41fb-90ee-255619e4c647 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:08.992881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fde2f-7b53-4488-9e75-1961665ebb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:08.993567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fde2f-7b53-4488-9e75-1961665ebb18 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:08.993908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fde2f-7b53-4488-9e75-1961665ebb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:08.994246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fde2f-7b53-4488-9e75-1961665ebb18 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:08.994940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49fde2f-7b53-4488-9e75-1961665ebb18 None admin] RBAC: Adding query filter params (type=image) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:08.995310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49fde2f-7b53-4488-9e75-1961665ebb18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:08.996783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49fde2f-7b53-4488-9e75-1961665ebb18 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:09.001349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 487/1001] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:08 2022] GET /identity/v3/services?type=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:09.758676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d37428-ffdb-43a6-aa3a-0f59a97e38fa None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:09.759195 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d37428-ffdb-43a6-aa3a-0f59a97e38fa None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:09.759306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d37428-ffdb-43a6-aa3a-0f59a97e38fa None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:09.760626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 515/1002] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:09.765722 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da074111-ec65-4954-82ff-ec5e8594b1c1 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:09.766132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da074111-ec65-4954-82ff-ec5e8594b1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:09.766315 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da074111-ec65-4954-82ff-ec5e8594b1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:09.778627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:09.785505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-da074111-ec65-4954-82ff-ec5e8594b1c1 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:09.786707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-da074111-ec65-4954-82ff-ec5e8594b1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:09.835452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5030-c793-4909-8b20-81ef6e090eb1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:09.835873 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5030-c793-4909-8b20-81ef6e090eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:09.836235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5030-c793-4909-8b20-81ef6e090eb1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:09.837636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 516/1003] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:16:09.842576 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efaf1a-9be3-4d4f-9594-440d4fe013e2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:09.843061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efaf1a-9be3-4d4f-9594-440d4fe013e2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:09.843534 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efaf1a-9be3-4d4f-9594-440d4fe013e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:09.860339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 488/1004] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:09.863635 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b7efaf1a-9be3-4d4f-9594-440d4fe013e2 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:09.865733 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b7efaf1a-9be3-4d4f-9594-440d4fe013e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:09.876724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:09.948659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 517/1005] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:16:10.130088 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da074111-ec65-4954-82ff-ec5e8594b1c1 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:10.217836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efaf1a-9be3-4d4f-9594-440d4fe013e2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:10.232635 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da074111-ec65-4954-82ff-ec5e8594b1c1 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:10.271807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c2d23-c82f-4fd0-acdb-119f694b0899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:10.272509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c2d23-c82f-4fd0-acdb-119f694b0899 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:10.273244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c2d23-c82f-4fd0-acdb-119f694b0899 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:10.273528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c2d23-c82f-4fd0-acdb-119f694b0899 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:10.274283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c2d23-c82f-4fd0-acdb-119f694b0899 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:10.275692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c2d23-c82f-4fd0-acdb-119f694b0899 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:10.278818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-6f8c2d23-c82f-4fd0-acdb-119f694b0899 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:10.290597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:10.291694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:10.292997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 489/1006] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Sep 12 23:16:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:10.297958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c2d23-c82f-4fd0-acdb-119f694b0899 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:10.331954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efaf1a-9be3-4d4f-9594-440d4fe013e2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:10.367897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cc155-e031-413d-8b01-28654cfaced7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:10.368534 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cc155-e031-413d-8b01-28654cfaced7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:10.368816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cc155-e031-413d-8b01-28654cfaced7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:10.369090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cc155-e031-413d-8b01-28654cfaced7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:10.369677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cc155-e031-413d-8b01-28654cfaced7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:10.371070 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cc155-e031-413d-8b01-28654cfaced7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-386cc155-e031-413d-8b01-28654cfaced7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:10.380246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:10.381262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:10.382097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 518/1007] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:16:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:10.386997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cc155-e031-413d-8b01-28654cfaced7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:10.398438 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:10.406588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c2d23-c82f-4fd0-acdb-119f694b0899 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:10.442090 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44f830-de40-4506-b197-bb24eff6c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:10.442769 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44f830-de40-4506-b197-bb24eff6c00a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:10.443058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44f830-de40-4506-b197-bb24eff6c00a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:10.443884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44f830-de40-4506-b197-bb24eff6c00a None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:10.444571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44f830-de40-4506-b197-bb24eff6c00a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:10.444876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44f830-de40-4506-b197-bb24eff6c00a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:10.446250 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44f830-de40-4506-b197-bb24eff6c00a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:10.451119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 490/1008] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Sep 12 23:16:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:10.456595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44f830-de40-4506-b197-bb24eff6c00a None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:10.501537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cc155-e031-413d-8b01-28654cfaced7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:10.523169 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:10.538779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56bb2-7c63-4338-8f39-d3089fd52c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:10.539389 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56bb2-7c63-4338-8f39-d3089fd52c63 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:10.539698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56bb2-7c63-4338-8f39-d3089fd52c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:10.539983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56bb2-7c63-4338-8f39-d3089fd52c63 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:10.540607 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed56bb2-7c63-4338-8f39-d3089fd52c63 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:10.540912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed56bb2-7c63-4338-8f39-d3089fd52c63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:10.542267 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed56bb2-7c63-4338-8f39-d3089fd52c63 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:10.547831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 519/1009] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Sep 12 23:16:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:10.554020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56bb2-7c63-4338-8f39-d3089fd52c63 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:10.563699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44f830-de40-4506-b197-bb24eff6c00a None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:10.599906 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcfe62-748b-4752-adbb-f9da68869e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:10.600540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcfe62-748b-4752-adbb-f9da68869e62 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:10.600825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcfe62-748b-4752-adbb-f9da68869e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:10.601101 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcfe62-748b-4752-adbb-f9da68869e62 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:10.601735 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcfe62-748b-4752-adbb-f9da68869e62 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:10.602039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcfe62-748b-4752-adbb-f9da68869e62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:10.603415 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcfe62-748b-4752-adbb-f9da68869e62 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:10.608860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 491/1010] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Sep 12 23:16:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:10.614375 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcfe62-748b-4752-adbb-f9da68869e62 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:10.637799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:10.671629 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56bb2-7c63-4338-8f39-d3089fd52c63 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:10.710262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dd1f4-2e84-496f-af36-db8315e2724f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:10.710971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dd1f4-2e84-496f-af36-db8315e2724f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:10.711352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dd1f4-2e84-496f-af36-db8315e2724f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:10.711650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dd1f4-2e84-496f-af36-db8315e2724f None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:10.712393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dd1f4-2e84-496f-af36-db8315e2724f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:10.712700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dd1f4-2e84-496f-af36-db8315e2724f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:10.714145 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dd1f4-2e84-496f-af36-db8315e2724f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:10.717882 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcfe62-748b-4752-adbb-f9da68869e62 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:10.724070 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 520/1011] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Sep 12 23:16:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:10.724742 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dd1f4-2e84-496f-af36-db8315e2724f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:10.745529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:10.754853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0692394-9a88-4551-9cfd-2ba0d6563428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:10.755472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0692394-9a88-4551-9cfd-2ba0d6563428 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:10.756328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0692394-9a88-4551-9cfd-2ba0d6563428 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:10.756707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0692394-9a88-4551-9cfd-2ba0d6563428 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:10.757304 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0692394-9a88-4551-9cfd-2ba0d6563428 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:10.758676 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0692394-9a88-4551-9cfd-2ba0d6563428 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:10.774987 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 492/1012] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:16:10 2022] POST /identity/v3/endpoints => generated 325 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:16:10.842459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dd1f4-2e84-496f-af36-db8315e2724f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:10.880278 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:10.887662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d12398-4539-4610-bab0-596a7d18d146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:10.888344 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d12398-4539-4610-bab0-596a7d18d146 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:10.888573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d12398-4539-4610-bab0-596a7d18d146 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:10.888849 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d12398-4539-4610-bab0-596a7d18d146 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:10.889526 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d12398-4539-4610-bab0-596a7d18d146 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:10.890912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d12398-4539-4610-bab0-596a7d18d146 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:10.918838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 521/1013] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:16:10 2022] POST /identity/v3/endpoints => generated 334 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:16:11.181856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aae390-acdd-43c8-9081-303285791496 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:11.182149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aae390-acdd-43c8-9081-303285791496 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:11.182428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aae390-acdd-43c8-9081-303285791496 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:11.183960 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 493/1014] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:11.189283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21e321-9b0d-42f8-a935-988efdabc5da None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:11.189757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21e321-9b0d-42f8-a935-988efdabc5da None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:11.190042 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21e321-9b0d-42f8-a935-988efdabc5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:11.209512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9f21e321-9b0d-42f8-a935-988efdabc5da None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:11.210712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9f21e321-9b0d-42f8-a935-988efdabc5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:11.287824 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 522/1015] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:11 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:11.560777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aae390-acdd-43c8-9081-303285791496 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:11.675206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aae390-acdd-43c8-9081-303285791496 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:11.716180 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284d1d6-f210-48de-a240-0ee106b0c6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:11.716794 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284d1d6-f210-48de-a240-0ee106b0c6e7 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:11.717075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284d1d6-f210-48de-a240-0ee106b0c6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:11.717353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284d1d6-f210-48de-a240-0ee106b0c6e7 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:11.718023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284d1d6-f210-48de-a240-0ee106b0c6e7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:11.719423 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284d1d6-f210-48de-a240-0ee106b0c6e7 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:11.741073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 494/1016] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Sep 12 23:16:11 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:16:12.986235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b909a67-9e98-48bc-9224-8db9127685ba None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:12.986762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b909a67-9e98-48bc-9224-8db9127685ba None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:12.986762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b909a67-9e98-48bc-9224-8db9127685ba None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:12.988184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 523/1017] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:12.993906 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8dcb9-7918-45ec-a576-cd20bea11386 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:12.994302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8dcb9-7918-45ec-a576-cd20bea11386 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:12.994480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8dcb9-7918-45ec-a576-cd20bea11386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:13.012742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-a1d8dcb9-7918-45ec-a576-cd20bea11386 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:13.013941 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a1d8dcb9-7918-45ec-a576-cd20bea11386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:13.061014 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:13.093019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 495/1018] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:13.216168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc8340-9c6f-4b80-9e3a-a5b502f8df71 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:13.216394 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc8340-9c6f-4b80-9e3a-a5b502f8df71 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:13.216628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc8340-9c6f-4b80-9e3a-a5b502f8df71 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:13.218121 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 524/1019] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:13.223024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8833fb5-e48c-4066-9c16-d2f7e6fc9ceb None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:13.223316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8833fb5-e48c-4066-9c16-d2f7e6fc9ceb None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:13.223612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8833fb5-e48c-4066-9c16-d2f7e6fc9ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:13.241972 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c8833fb5-e48c-4066-9c16-d2f7e6fc9ceb None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:13.243150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c8833fb5-e48c-4066-9c16-d2f7e6fc9ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:13.310625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 496/1020] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:13.376961 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc8340-9c6f-4b80-9e3a-a5b502f8df71 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:13.481218 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc8340-9c6f-4b80-9e3a-a5b502f8df71 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:13.495142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:13.516709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5760a6-536f-449a-a305-ec43a8e713e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:13.517340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5760a6-536f-449a-a305-ec43a8e713e0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:13.517620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5760a6-536f-449a-a305-ec43a8e713e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:13.517899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5760a6-536f-449a-a305-ec43a8e713e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:13.519010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5760a6-536f-449a-a305-ec43a8e713e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:13.520470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5760a6-536f-449a-a305-ec43a8e713e0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:13.522811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 525/1021] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:13.528355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8833fb5-e48c-4066-9c16-d2f7e6fc9ceb None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:13.582554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5760a6-536f-449a-a305-ec43a8e713e0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:13.635972 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8833fb5-e48c-4066-9c16-d2f7e6fc9ceb None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:13.639165 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:13.675590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c55ac-5290-45fb-a18f-10abefe5b99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:13.676201 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c55ac-5290-45fb-a18f-10abefe5b99b None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:13.676481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c55ac-5290-45fb-a18f-10abefe5b99b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:13.676752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c55ac-5290-45fb-a18f-10abefe5b99b None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:13.677923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c55ac-5290-45fb-a18f-10abefe5b99b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:13.679368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c55ac-5290-45fb-a18f-10abefe5b99b None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:13.684860 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5760a6-536f-449a-a305-ec43a8e713e0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:13.712735 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 497/1022] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Sep 12 23:16:13 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:16:13.723965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab6415-bd2b-42cf-952d-ce9806e1173b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:13.724773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab6415-bd2b-42cf-952d-ce9806e1173b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:13.725052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab6415-bd2b-42cf-952d-ce9806e1173b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:13.725325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab6415-bd2b-42cf-952d-ce9806e1173b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:13.738009 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab6415-bd2b-42cf-952d-ce9806e1173b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b0ab6415-bd2b-42cf-952d-ce9806e1173b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:16:13.743700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:13.744740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:13.748568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:13.748568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:13.748568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:13.748568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:13.748568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:16:13.748568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:16:13.748568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:16:13.748568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:16:13.748568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:16:13.748568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:16:13.748568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:16:13.748568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:13.748568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 526/1023] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Sep 12 23:16:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:13.751451 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c55ac-5290-45fb-a18f-10abefe5b99b None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:13.866228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c55ac-5290-45fb-a18f-10abefe5b99b None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:13.903087 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb36b9b-20cc-43d9-baa8-a4ee225d1d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:13.903319 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb36b9b-20cc-43d9-baa8-a4ee225d1d7d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:13.903598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb36b9b-20cc-43d9-baa8-a4ee225d1d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:13.904548 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb36b9b-20cc-43d9-baa8-a4ee225d1d7d None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:13.904548 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb36b9b-20cc-43d9-baa8-a4ee225d1d7d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:13.904844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb36b9b-20cc-43d9-baa8-a4ee225d1d7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:13.906270 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb36b9b-20cc-43d9-baa8-a4ee225d1d7d None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:13.916302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 498/1024] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Sep 12 23:16:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:13.923004 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab6415-bd2b-42cf-952d-ce9806e1173b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:13.937882 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc9ef0-b89b-41da-9b0e-05ba93ab05ae None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:13.938172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc9ef0-b89b-41da-9b0e-05ba93ab05ae None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:13.938450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc9ef0-b89b-41da-9b0e-05ba93ab05ae None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:13.940710 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 499/1025] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:16:13.945587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d332893-5e35-4396-a77a-6e0bd20038af None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:13.945883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d332893-5e35-4396-a77a-6e0bd20038af None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:13.946166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d332893-5e35-4396-a77a-6e0bd20038af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:13.965212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-4d332893-5e35-4396-a77a-6e0bd20038af None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:13.966415 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4d332893-5e35-4396-a77a-6e0bd20038af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:13.970929 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:13.982222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:14.033883 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab6415-bd2b-42cf-952d-ce9806e1173b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:14.042508 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 500/1026] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:16:14.070870 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4557b63-f627-45d1-b2b3-247a63176e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:14.071508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4557b63-f627-45d1-b2b3-247a63176e9e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:14.071817 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4557b63-f627-45d1-b2b3-247a63176e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:14.072093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4557b63-f627-45d1-b2b3-247a63176e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:14.082204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4557b63-f627-45d1-b2b3-247a63176e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:14.084250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4557b63-f627-45d1-b2b3-247a63176e9e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a4557b63-f627-45d1-b2b3-247a63176e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:16:14.098694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:14.100011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:16:14.101378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:16:14.101378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:14.101378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:14.101378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 527/1027] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:14.108252 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d332893-5e35-4396-a77a-6e0bd20038af None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:14.207591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d332893-5e35-4396-a77a-6e0bd20038af None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:14.241081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc017f5-7e00-40e4-8030-aef1ef4aa517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:14.241695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc017f5-7e00-40e4-8030-aef1ef4aa517 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:14.241975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc017f5-7e00-40e4-8030-aef1ef4aa517 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:14.242249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc017f5-7e00-40e4-8030-aef1ef4aa517 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:14.242883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc017f5-7e00-40e4-8030-aef1ef4aa517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:14.243186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc017f5-7e00-40e4-8030-aef1ef4aa517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:14.244591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc017f5-7e00-40e4-8030-aef1ef4aa517 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:14.261113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 501/1028] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:14.262395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4557b63-f627-45d1-b2b3-247a63176e9e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:14.326415 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc017f5-7e00-40e4-8030-aef1ef4aa517 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:14.338221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:14.362582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:14.371155 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4557b63-f627-45d1-b2b3-247a63176e9e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:14.405468 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3810d-fdff-4ec3-a50c-8a3aa9e2fe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:14.406098 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3810d-fdff-4ec3-a50c-8a3aa9e2fe3d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:14.406381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3810d-fdff-4ec3-a50c-8a3aa9e2fe3d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:14.406664 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3810d-fdff-4ec3-a50c-8a3aa9e2fe3d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:14.414399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3810d-fdff-4ec3-a50c-8a3aa9e2fe3d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:14.415869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3810d-fdff-4ec3-a50c-8a3aa9e2fe3d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-25b3810d-fdff-4ec3-a50c-8a3aa9e2fe3d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:16:14.428564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:14.429614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:14.430600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:14.430600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:14.430600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:16:14.430600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:16:14.430600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:16:14.430600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:16:14.430600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:16:14.430600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:16:14.430600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:16:14.430600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:14.430600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 528/1029] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Sep 12 23:16:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:14.435514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3810d-fdff-4ec3-a50c-8a3aa9e2fe3d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:14.440865 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc017f5-7e00-40e4-8030-aef1ef4aa517 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:14.479523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a45325-1233-4ff6-bf81-ec8fc9f0b9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:14.480571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a45325-1233-4ff6-bf81-ec8fc9f0b9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:14.481016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a45325-1233-4ff6-bf81-ec8fc9f0b9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:14.481183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a45325-1233-4ff6-bf81-ec8fc9f0b9c2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:14.481781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a45325-1233-4ff6-bf81-ec8fc9f0b9c2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:14.483146 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a45325-1233-4ff6-bf81-ec8fc9f0b9c2 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-73a45325-1233-4ff6-bf81-ec8fc9f0b9c2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:14.492652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:14.493674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:14.494513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:14.494513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:14.494513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:14.494513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:14.494513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:14.494513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:14.494513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:14.494513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:14.494513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:14.494513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:14.494513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:16:14.494513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:14.494513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:16:14.494513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:14.498882 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 502/1030] 10.222.0.27 () {56 vars in 1105 bytes} [Mon Sep 12 23:16:14 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:14.499753 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a45325-1233-4ff6-bf81-ec8fc9f0b9c2 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:14.547173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3810d-fdff-4ec3-a50c-8a3aa9e2fe3d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:14.577065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:14.586939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150706e-7cc9-45c9-a8f9-a26e32fe625a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:14.587550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150706e-7cc9-45c9-a8f9-a26e32fe625a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:14.587857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150706e-7cc9-45c9-a8f9-a26e32fe625a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:14.588133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150706e-7cc9-45c9-a8f9-a26e32fe625a None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:14.588922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150706e-7cc9-45c9-a8f9-a26e32fe625a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:14.589228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150706e-7cc9-45c9-a8f9-a26e32fe625a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:14.590652 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150706e-7cc9-45c9-a8f9-a26e32fe625a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:14.608782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 529/1031] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Sep 12 23:16:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:14.609880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a45325-1233-4ff6-bf81-ec8fc9f0b9c2 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:14.614201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150706e-7cc9-45c9-a8f9-a26e32fe625a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:14.645204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:14.663328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a128036-207a-4d6b-bae5-fc060a5883fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:14.666039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a128036-207a-4d6b-bae5-fc060a5883fe None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:14.666326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a128036-207a-4d6b-bae5-fc060a5883fe None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:14.666530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a128036-207a-4d6b-bae5-fc060a5883fe None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:14.667171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a128036-207a-4d6b-bae5-fc060a5883fe None admin] RBAC: Adding query filter params (name=image) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:14.667470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a128036-207a-4d6b-bae5-fc060a5883fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:14.668963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a128036-207a-4d6b-bae5-fc060a5883fe None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:14.675644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 503/1032] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:14.681483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a128036-207a-4d6b-bae5-fc060a5883fe None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:14.748940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150706e-7cc9-45c9-a8f9-a26e32fe625a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:14.784899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ade55d-6728-417e-a8fc-9f821eacd410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:14.785446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ade55d-6728-417e-a8fc-9f821eacd410 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:14.785731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ade55d-6728-417e-a8fc-9f821eacd410 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:14.786000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ade55d-6728-417e-a8fc-9f821eacd410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:14.788612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a128036-207a-4d6b-bae5-fc060a5883fe None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:14.822224 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b462528-0fa1-497a-9eff-6091bf67cc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:14.822828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b462528-0fa1-497a-9eff-6091bf67cc61 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:14.823109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b462528-0fa1-497a-9eff-6091bf67cc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:14.823384 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b462528-0fa1-497a-9eff-6091bf67cc61 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:14.824040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b462528-0fa1-497a-9eff-6091bf67cc61 None admin] RBAC: Adding query filter params (type=image) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:14.824345 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b462528-0fa1-497a-9eff-6091bf67cc61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:14.825718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b462528-0fa1-497a-9eff-6091bf67cc61 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:14.828356 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:14.836660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 504/1033] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:14.837952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b462528-0fa1-497a-9eff-6091bf67cc61 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:14.861040 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ade55d-6728-417e-a8fc-9f821eacd410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:14.862606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ade55d-6728-417e-a8fc-9f821eacd410 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:14.877170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:14.943530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b462528-0fa1-497a-9eff-6091bf67cc61 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:14.977883 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 530/1034] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:16:14 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:16:14.982221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90b0d0-e80d-41e5-a0aa-98f10a3ae18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:14.982929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90b0d0-e80d-41e5-a0aa-98f10a3ae18f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:14.983215 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90b0d0-e80d-41e5-a0aa-98f10a3ae18f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:14.983491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90b0d0-e80d-41e5-a0aa-98f10a3ae18f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:14.985633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90b0d0-e80d-41e5-a0aa-98f10a3ae18f None admin] RBAC: Adding query filter params (interface=public, service_id=d134255540d84ae8ba0779e13d51a368, region_id=RegionOne) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:14.985633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90b0d0-e80d-41e5-a0aa-98f10a3ae18f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:14.986139 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90b0d0-e80d-41e5-a0aa-98f10a3ae18f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:14.988607 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ade55d-6728-417e-a8fc-9f821eacd410 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:14.993419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 505/1035] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Sep 12 23:16:14 2022] GET /identity/v3/endpoints?service_id=d134255540d84ae8ba0779e13d51a368&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:14.998997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90b0d0-e80d-41e5-a0aa-98f10a3ae18f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:15.064511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:15.094412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ade55d-6728-417e-a8fc-9f821eacd410 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:15.105558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90b0d0-e80d-41e5-a0aa-98f10a3ae18f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:15.108392 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:15.132253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe1226-ed84-4c68-b190-606bcc7a82c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:15.132881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe1226-ed84-4c68-b190-606bcc7a82c9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:15.133165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe1226-ed84-4c68-b190-606bcc7a82c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:15.133441 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe1226-ed84-4c68-b190-606bcc7a82c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:15.143725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d6245-610c-4655-b339-82f8747071ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:15.144212 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe1226-ed84-4c68-b190-606bcc7a82c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:15.144319 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d6245-610c-4655-b339-82f8747071ad None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:15.144596 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d6245-610c-4655-b339-82f8747071ad None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:15.144867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d6245-610c-4655-b339-82f8747071ad None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:15.145466 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d6245-610c-4655-b339-82f8747071ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:15.145637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe1226-ed84-4c68-b190-606bcc7a82c9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:15.147353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d6245-610c-4655-b339-82f8747071ad None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:15.154042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 506/1036] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Sep 12 23:16:14 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-61fe1226-ed84-4c68-b190-606bcc7a82c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:16:15.161328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:15.162416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:16:15.168700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:16:15.168700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:15.168700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:15.168700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 531/1037] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:15.168942 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d6245-610c-4655-b339-82f8747071ad None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:15.284129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d6245-610c-4655-b339-82f8747071ad None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:15.312656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:15.322477 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db2543-a50b-417e-88d5-a1b253f7518a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:15.323087 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db2543-a50b-417e-88d5-a1b253f7518a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:15.323368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db2543-a50b-417e-88d5-a1b253f7518a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:15.323672 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db2543-a50b-417e-88d5-a1b253f7518a None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:15.324321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db2543-a50b-417e-88d5-a1b253f7518a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:15.324626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db2543-a50b-417e-88d5-a1b253f7518a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:15.326005 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db2543-a50b-417e-88d5-a1b253f7518a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:15.338812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 507/1038] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:15.344746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe1226-ed84-4c68-b190-606bcc7a82c9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:15.448287 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe1226-ed84-4c68-b190-606bcc7a82c9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:15.482177 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c4c99-175c-4677-bd35-4f8aaf0c2565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:15.482816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c4c99-175c-4677-bd35-4f8aaf0c2565 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:15.483091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c4c99-175c-4677-bd35-4f8aaf0c2565 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:15.483365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c4c99-175c-4677-bd35-4f8aaf0c2565 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:15.490581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c4c99-175c-4677-bd35-4f8aaf0c2565 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:15.492102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c4c99-175c-4677-bd35-4f8aaf0c2565 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-fa2c4c99-175c-4677-bd35-4f8aaf0c2565 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:16:15.504110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:15.505016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:15.506493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:16:15.506493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:16:15.506493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:16:15.506493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:16:15.506493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:16:15.506493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:16:15.506493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:16:15.506493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:15.506493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 532/1039] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:16:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:15.511848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db2543-a50b-417e-88d5-a1b253f7518a None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:15.616345 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db2543-a50b-417e-88d5-a1b253f7518a None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:15.652141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893d201-f266-405b-a9fd-c8a81d6a6caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:15.652754 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d201-f266-405b-a9fd-c8a81d6a6caa None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:15.653035 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d201-f266-405b-a9fd-c8a81d6a6caa None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:15.653305 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d201-f266-405b-a9fd-c8a81d6a6caa None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:15.653996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893d201-f266-405b-a9fd-c8a81d6a6caa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:15.654299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893d201-f266-405b-a9fd-c8a81d6a6caa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:15.655774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893d201-f266-405b-a9fd-c8a81d6a6caa None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:15.667771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 508/1040] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:16:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:15.673514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c4c99-175c-4677-bd35-4f8aaf0c2565 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:15.699112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:15.772800 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c4c99-175c-4677-bd35-4f8aaf0c2565 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:15.806530 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443af4bc-fe2c-49c7-9eaf-b71a389d0c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:15.807282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443af4bc-fe2c-49c7-9eaf-b71a389d0c0b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:15.807692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443af4bc-fe2c-49c7-9eaf-b71a389d0c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:15.808065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443af4bc-fe2c-49c7-9eaf-b71a389d0c0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:15.808906 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443af4bc-fe2c-49c7-9eaf-b71a389d0c0b None admin] RBAC: Adding query filter params (role.id=9940e26019d341f4a9af2fa399a02bc8, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=f5fa28ffdafd435cb2d93bd453add3cb) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:15.809312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443af4bc-fe2c-49c7-9eaf-b71a389d0c0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:15.810816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443af4bc-fe2c-49c7-9eaf-b71a389d0c0b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:15.816524 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 533/1041] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:16:15 2022] GET /identity/v3/role_assignments?user.id=f5fa28ffdafd435cb2d93bd453add3cb&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=9940e26019d341f4a9af2fa399a02bc8 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:15.995121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686431f-f65a-4e01-8dd5-b0fcaf9ed66c None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:15.995408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686431f-f65a-4e01-8dd5-b0fcaf9ed66c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:15.995712 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686431f-f65a-4e01-8dd5-b0fcaf9ed66c None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:15.997110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 509/1042] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:16.002468 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651735f4-7351-432f-9453-db0174d11ab9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:16.002756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651735f4-7351-432f-9453-db0174d11ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:16.003043 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651735f4-7351-432f-9453-db0174d11ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:16.025328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-651735f4-7351-432f-9453-db0174d11ab9 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:16.026522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-651735f4-7351-432f-9453-db0174d11ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:16.098873 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 534/1043] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:16.379333 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686431f-f65a-4e01-8dd5-b0fcaf9ed66c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:16.435014 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:16.490557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686431f-f65a-4e01-8dd5-b0fcaf9ed66c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:16.524166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1c3d2-f3b8-4320-a25b-6e413eb9f9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:16.524793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1c3d2-f3b8-4320-a25b-6e413eb9f9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:16.525069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1c3d2-f3b8-4320-a25b-6e413eb9f9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:16.525341 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1c3d2-f3b8-4320-a25b-6e413eb9f9d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:16.526437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1c3d2-f3b8-4320-a25b-6e413eb9f9d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:16.530917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1c3d2-f3b8-4320-a25b-6e413eb9f9d6 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:16.537669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 510/1044] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:16.537827 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651735f4-7351-432f-9453-db0174d11ab9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:16.554281 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:16.636702 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651735f4-7351-432f-9453-db0174d11ab9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:16.680037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706cc2fa-0b5d-4b0d-9f64-06fdd8123d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:16.680757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cc2fa-0b5d-4b0d-9f64-06fdd8123d18 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:16.681040 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cc2fa-0b5d-4b0d-9f64-06fdd8123d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:16.681314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cc2fa-0b5d-4b0d-9f64-06fdd8123d18 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:16.681970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706cc2fa-0b5d-4b0d-9f64-06fdd8123d18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:16.683395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706cc2fa-0b5d-4b0d-9f64-06fdd8123d18 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:16.715595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 535/1045] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:16:16 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:16:17.366114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64046d21-8ea2-473c-8330-f33ffc9e357d None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:17.366302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64046d21-8ea2-473c-8330-f33ffc9e357d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:17.366588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64046d21-8ea2-473c-8330-f33ffc9e357d None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:17.368102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 511/1046] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:17.373085 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb457d0-54bb-4f56-b758-418458e7deca None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:17.373557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb457d0-54bb-4f56-b758-418458e7deca None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:17.373647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb457d0-54bb-4f56-b758-418458e7deca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:17.392246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8fb457d0-54bb-4f56-b758-418458e7deca None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:17.393435 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8fb457d0-54bb-4f56-b758-418458e7deca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:17.467614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 536/1047] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:17.732403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64046d21-8ea2-473c-8330-f33ffc9e357d None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:17.836775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64046d21-8ea2-473c-8330-f33ffc9e357d None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:17.852681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:17.871278 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0a2bc-f2c6-4f1d-b7ff-9e2770451a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:17.871938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0a2bc-f2c6-4f1d-b7ff-9e2770451a65 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:17.872222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0a2bc-f2c6-4f1d-b7ff-9e2770451a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:17.872496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0a2bc-f2c6-4f1d-b7ff-9e2770451a65 None admin] PATH_INFO: `/v3/services/image` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:17.873079 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0a2bc-f2c6-4f1d-b7ff-9e2770451a65 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:17.874454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0a2bc-f2c6-4f1d-b7ff-9e2770451a65 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-b7e0a2bc-f2c6-4f1d-b7ff-9e2770451a65 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:17.884360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:17.885467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:17.886424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:17.886424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:17.886424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:17.886424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:17.886424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:17.886424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:17.886424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:17.886424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:17.886424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:17.886424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:17.886424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:16:17.886424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:16:17.886424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:16:17.886424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:17.887939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 512/1048] 10.222.0.27 () {56 vars in 1105 bytes} [Mon Sep 12 23:16:17 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:17.891838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb457d0-54bb-4f56-b758-418458e7deca None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:17.895898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:18.006783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1849d9e-ff3b-429b-a5bf-ee8b77b9a415 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:18.006970 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1849d9e-ff3b-429b-a5bf-ee8b77b9a415 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:18.007261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1849d9e-ff3b-429b-a5bf-ee8b77b9a415 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:18.008305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb457d0-54bb-4f56-b758-418458e7deca None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:18.008630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 513/1049] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:16:18.013649 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca99847-a8c0-4a79-8985-b0734360d64e None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:18.013932 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca99847-a8c0-4a79-8985-b0734360d64e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:18.014210 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca99847-a8c0-4a79-8985-b0734360d64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:18.032498 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-dca99847-a8c0-4a79-8985-b0734360d64e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:18.033701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-dca99847-a8c0-4a79-8985-b0734360d64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:18.042646 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d54292-0b8e-4ee5-a1c2-c32c0ed9ea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:18.043256 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d54292-0b8e-4ee5-a1c2-c32c0ed9ea7b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:18.043534 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d54292-0b8e-4ee5-a1c2-c32c0ed9ea7b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:18.043829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d54292-0b8e-4ee5-a1c2-c32c0ed9ea7b None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:18.044459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d54292-0b8e-4ee5-a1c2-c32c0ed9ea7b None admin] RBAC: Adding query filter params (name=image) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:18.044759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d54292-0b8e-4ee5-a1c2-c32c0ed9ea7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:18.046113 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d54292-0b8e-4ee5-a1c2-c32c0ed9ea7b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:18.055993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 537/1050] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:18.056987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d54292-0b8e-4ee5-a1c2-c32c0ed9ea7b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:18.102183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 514/1051] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:16:18.126138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:18.161391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d54292-0b8e-4ee5-a1c2-c32c0ed9ea7b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:18.196254 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0be05-490c-4678-ad59-c77ccf021437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:18.196856 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0be05-490c-4678-ad59-c77ccf021437 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:18.197130 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0be05-490c-4678-ad59-c77ccf021437 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:18.197396 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0be05-490c-4678-ad59-c77ccf021437 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:18.198008 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0be05-490c-4678-ad59-c77ccf021437 None admin] RBAC: Adding query filter params (type=image) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:18.198305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0be05-490c-4678-ad59-c77ccf021437 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:18.199717 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0be05-490c-4678-ad59-c77ccf021437 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:18.204674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 538/1052] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:18.210708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca99847-a8c0-4a79-8985-b0734360d64e None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:18.309090 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca99847-a8c0-4a79-8985-b0734360d64e None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:18.319234 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:18.343792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a5314-f4bd-4d5f-bafb-7b5b3f6d11a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:18.344398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a5314-f4bd-4d5f-bafb-7b5b3f6d11a6 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:18.344675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a5314-f4bd-4d5f-bafb-7b5b3f6d11a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:18.344950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a5314-f4bd-4d5f-bafb-7b5b3f6d11a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:18.345623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a5314-f4bd-4d5f-bafb-7b5b3f6d11a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:18.346990 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a5314-f4bd-4d5f-bafb-7b5b3f6d11a6 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:18.371602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 515/1053] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:16:18 2022] POST /identity/v3/endpoints => generated 327 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:16:18.378033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0be05-490c-4678-ad59-c77ccf021437 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:18.488297 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0be05-490c-4678-ad59-c77ccf021437 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:18.520753 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:18.534445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e8846-f8e4-49b3-8cef-311c237e4efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:18.534807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8846-f8e4-49b3-8cef-311c237e4efd None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:18.535007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8846-f8e4-49b3-8cef-311c237e4efd None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:18.535280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8846-f8e4-49b3-8cef-311c237e4efd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:18.547332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e8846-f8e4-49b3-8cef-311c237e4efd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-605e8846-f8e4-49b3-8cef-311c237e4efd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:16:18.552315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:18.553251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:18.554109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:18.554109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:18.554109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:18.554109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:18.554109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:16:18.554109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:16:18.554109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:16:18.554109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:16:18.554109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:16:18.554109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:16:18.554109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:16:18.554109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:18.554109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 539/1054] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Sep 12 23:16:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:18.559645 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a5314-f4bd-4d5f-bafb-7b5b3f6d11a6 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:18.666774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a5314-f4bd-4d5f-bafb-7b5b3f6d11a6 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:18.700798 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09771553-4f9f-4906-b01e-722563692674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:18.701407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09771553-4f9f-4906-b01e-722563692674 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:18.701687 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09771553-4f9f-4906-b01e-722563692674 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:18.701961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09771553-4f9f-4906-b01e-722563692674 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:18.702618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09771553-4f9f-4906-b01e-722563692674 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:18.702920 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09771553-4f9f-4906-b01e-722563692674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:18.704321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09771553-4f9f-4906-b01e-722563692674 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:18.717031 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 516/1055] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Sep 12 23:16:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:18.718586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e8846-f8e4-49b3-8cef-311c237e4efd None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:18.821741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e8846-f8e4-49b3-8cef-311c237e4efd None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:18.860872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f5767-4c74-4070-9ab8-b388b0234df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:18.861519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f5767-4c74-4070-9ab8-b388b0234df6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:18.861798 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f5767-4c74-4070-9ab8-b388b0234df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:18.862068 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f5767-4c74-4070-9ab8-b388b0234df6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:18.872967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f5767-4c74-4070-9ab8-b388b0234df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:18.874420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f5767-4c74-4070-9ab8-b388b0234df6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-450f5767-4c74-4070-9ab8-b388b0234df6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:16:18.889768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:18.890768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:16:18.897032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:16:18.897032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:18.897032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:18.897032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 540/1056] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:18.898855 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09771553-4f9f-4906-b01e-722563692674 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:18.929203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:18.953153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a50e3-8be8-4a72-af9e-470e376b9462 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:18.953498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a50e3-8be8-4a72-af9e-470e376b9462 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:18.953999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a50e3-8be8-4a72-af9e-470e376b9462 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:18.955487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 541/1057] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:16:18.960576 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751148a-dba7-4855-862d-d5d9000206ba None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:18.960857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751148a-dba7-4855-862d-d5d9000206ba None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:18.961133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751148a-dba7-4855-862d-d5d9000206ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:18.979230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1751148a-dba7-4855-862d-d5d9000206ba None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:18.980470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1751148a-dba7-4855-862d-d5d9000206ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:19.012108 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09771553-4f9f-4906-b01e-722563692674 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:19.016150 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:19.051350 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99356722-6ad3-43e2-a015-91628feb2c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:19.051643 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 542/1058] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:16:19.052270 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99356722-6ad3-43e2-a015-91628feb2c74 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:19.052611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99356722-6ad3-43e2-a015-91628feb2c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:19.052945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99356722-6ad3-43e2-a015-91628feb2c74 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:19.053650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99356722-6ad3-43e2-a015-91628feb2c74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:19.054013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99356722-6ad3-43e2-a015-91628feb2c74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:19.055440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99356722-6ad3-43e2-a015-91628feb2c74 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:19.070382 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 517/1059] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:19.076617 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751148a-dba7-4855-862d-d5d9000206ba None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:19.177635 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751148a-dba7-4855-862d-d5d9000206ba None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:19.211212 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8182c02-d2be-4ac3-893e-cbd40aee5647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:19.211871 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8182c02-d2be-4ac3-893e-cbd40aee5647 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:19.212155 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8182c02-d2be-4ac3-893e-cbd40aee5647 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:19.212429 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8182c02-d2be-4ac3-893e-cbd40aee5647 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:19.219317 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8182c02-d2be-4ac3-893e-cbd40aee5647 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:19.220782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8182c02-d2be-4ac3-893e-cbd40aee5647 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d8182c02-d2be-4ac3-893e-cbd40aee5647 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:16:19.232246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:19.233176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:19.234206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:19.234206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:19.234206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:16:19.234206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:16:19.234206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:16:19.234206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:16:19.234206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:16:19.234206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:16:19.234206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:16:19.234206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:19.235990 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 543/1060] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Sep 12 23:16:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:19.239156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99356722-6ad3-43e2-a015-91628feb2c74 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:19.324705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8182c02-d2be-4ac3-893e-cbd40aee5647 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:19.335911 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:19.351504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99356722-6ad3-43e2-a015-91628feb2c74 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:19.363340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:19.386494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655e32b-1304-491e-8739-9fe7b105a94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:19.387097 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655e32b-1304-491e-8739-9fe7b105a94b None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:19.387376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655e32b-1304-491e-8739-9fe7b105a94b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:19.387673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655e32b-1304-491e-8739-9fe7b105a94b None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:19.388395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655e32b-1304-491e-8739-9fe7b105a94b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:19.388695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655e32b-1304-491e-8739-9fe7b105a94b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:19.390114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655e32b-1304-491e-8739-9fe7b105a94b None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:19.413318 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 518/1061] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Sep 12 23:16:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:19.414321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655e32b-1304-491e-8739-9fe7b105a94b None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:19.440679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8182c02-d2be-4ac3-893e-cbd40aee5647 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:19.478009 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68798327-9e29-4233-b37a-0c953ecb74d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:19.478629 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68798327-9e29-4233-b37a-0c953ecb74d4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:19.478910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68798327-9e29-4233-b37a-0c953ecb74d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:19.479183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68798327-9e29-4233-b37a-0c953ecb74d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:19.491679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68798327-9e29-4233-b37a-0c953ecb74d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-68798327-9e29-4233-b37a-0c953ecb74d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:16:19.496848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:19.497785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:19.498639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:19.498639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:19.498639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:19.498639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:19.498639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:16:19.498639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:16:19.498639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:16:19.498639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:16:19.498639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:16:19.498639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:16:19.498639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:16:19.498639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:19.498639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 544/1062] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:16:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:19.504058 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68798327-9e29-4233-b37a-0c953ecb74d4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:19.516752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655e32b-1304-491e-8739-9fe7b105a94b None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:19.552877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c9e3b-1655-487d-a044-ee9c05a3f4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:19.553519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c9e3b-1655-487d-a044-ee9c05a3f4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:19.553843 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c9e3b-1655-487d-a044-ee9c05a3f4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:19.554146 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c9e3b-1655-487d-a044-ee9c05a3f4d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:19.564299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c9e3b-1655-487d-a044-ee9c05a3f4d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:19.565739 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c9e3b-1655-487d-a044-ee9c05a3f4d9 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:19.572702 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-248c9e3b-1655-487d-a044-ee9c05a3f4d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:16:19.582348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:19.583346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:16:19.584974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:16:19.584974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:19.584974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:19.584974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 519/1063] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:19.589495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c9e3b-1655-487d-a044-ee9c05a3f4d9 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:19.606175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:19.617285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68798327-9e29-4233-b37a-0c953ecb74d4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:19.652821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08c97f-8989-4320-af4d-8df51e6600ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:19.653436 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08c97f-8989-4320-af4d-8df51e6600ed None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:19.653717 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08c97f-8989-4320-af4d-8df51e6600ed None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:19.654059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08c97f-8989-4320-af4d-8df51e6600ed None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:19.654720 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08c97f-8989-4320-af4d-8df51e6600ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:19.655050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08c97f-8989-4320-af4d-8df51e6600ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:19.656431 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08c97f-8989-4320-af4d-8df51e6600ed None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:19.666480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 545/1064] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:16:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:19.673307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08c97f-8989-4320-af4d-8df51e6600ed None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:19.705415 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c9e3b-1655-487d-a044-ee9c05a3f4d9 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:19.740931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c521b-89cb-404e-b328-a6da08c6a170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:19.741533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c521b-89cb-404e-b328-a6da08c6a170 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:19.741814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c521b-89cb-404e-b328-a6da08c6a170 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:19.742090 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c521b-89cb-404e-b328-a6da08c6a170 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:19.742892 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c521b-89cb-404e-b328-a6da08c6a170 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:19.743197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c521b-89cb-404e-b328-a6da08c6a170 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:19.745135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c521b-89cb-404e-b328-a6da08c6a170 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:19.762630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 520/1065] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:19.764008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c521b-89cb-404e-b328-a6da08c6a170 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:19.777407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08c97f-8989-4320-af4d-8df51e6600ed None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:19.812743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63071e06-9738-416e-a53b-eab890cc8bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:19.813368 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63071e06-9738-416e-a53b-eab890cc8bfc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:19.813645 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63071e06-9738-416e-a53b-eab890cc8bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:19.813913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63071e06-9738-416e-a53b-eab890cc8bfc None admin] PATH_INFO: `/v3/users/12b3998d0e044b8cba6f29bd8e9bcff7` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:19.826917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:19.828091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63071e06-9738-416e-a53b-eab890cc8bfc None admin] RBAC: Authorizing `identity:get_user(user_id=12b3998d0e044b8cba6f29bd8e9bcff7)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:19.829873 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63071e06-9738-416e-a53b-eab890cc8bfc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:19.836265 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 546/1066] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:16:19 2022] GET /identity/v3/users/12b3998d0e044b8cba6f29bd8e9bcff7 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:19.837092 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63071e06-9738-416e-a53b-eab890cc8bfc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:19.849154 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:19.874594 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c521b-89cb-404e-b328-a6da08c6a170 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:19.909318 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc80d0a-bb64-4b39-82ab-ba0794ff7441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:19.909934 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc80d0a-bb64-4b39-82ab-ba0794ff7441 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:19.910218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc80d0a-bb64-4b39-82ab-ba0794ff7441 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:19.910528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc80d0a-bb64-4b39-82ab-ba0794ff7441 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:19.923733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc80d0a-bb64-4b39-82ab-ba0794ff7441 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:19.925192 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc80d0a-bb64-4b39-82ab-ba0794ff7441 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-cdc80d0a-bb64-4b39-82ab-ba0794ff7441 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:16:19.938116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:19.939024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:19.940204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:16:19.940204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:16:19.940204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:16:19.940204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:16:19.940204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:16:19.940204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:16:19.940204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:16:19.940204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:19.940204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 521/1067] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:16:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:19.948365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc80d0a-bb64-4b39-82ab-ba0794ff7441 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:19.958096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63071e06-9738-416e-a53b-eab890cc8bfc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:19.995124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad2ab8-e031-4458-977f-7f08b6570394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:19.995761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad2ab8-e031-4458-977f-7f08b6570394 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:19.996046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad2ab8-e031-4458-977f-7f08b6570394 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:19.996313 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad2ab8-e031-4458-977f-7f08b6570394 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:20.053617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc80d0a-bb64-4b39-82ab-ba0794ff7441 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:20.066519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad2ab8-e031-4458-977f-7f08b6570394 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:20.068848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad2ab8-e031-4458-977f-7f08b6570394 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:20.074614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:20.087107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:20.105066 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792018c4-49e7-4d34-845d-ffdbb3fa868d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:20.105990 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792018c4-49e7-4d34-845d-ffdbb3fa868d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:20.106495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792018c4-49e7-4d34-845d-ffdbb3fa868d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:20.106912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792018c4-49e7-4d34-845d-ffdbb3fa868d None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:20.107855 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792018c4-49e7-4d34-845d-ffdbb3fa868d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:20.108168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792018c4-49e7-4d34-845d-ffdbb3fa868d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:20.109610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792018c4-49e7-4d34-845d-ffdbb3fa868d None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:20.123205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 522/1068] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:16:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:20.129312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792018c4-49e7-4d34-845d-ffdbb3fa868d None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:20.186257 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 547/1069] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:16:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 23:16:20.187311 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad2ab8-e031-4458-977f-7f08b6570394 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:20.236213 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792018c4-49e7-4d34-845d-ffdbb3fa868d None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:20.271773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1743bf-9209-4bba-a5dd-82ffaee999f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:20.272427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1743bf-9209-4bba-a5dd-82ffaee999f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:20.272701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1743bf-9209-4bba-a5dd-82ffaee999f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:20.272972 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1743bf-9209-4bba-a5dd-82ffaee999f3 None admin] PATH_INFO: `/v3/projects/32bbb936501846e8914acf07890c2891/users/f5fa28ffdafd435cb2d93bd453add3cb/roles/9940e26019d341f4a9af2fa399a02bc8` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:20.275117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1743bf-9209-4bba-a5dd-82ffaee999f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=32bbb936501846e8914acf07890c2891, user_id=f5fa28ffdafd435cb2d93bd453add3cb, role_id=9940e26019d341f4a9af2fa399a02bc8)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:20.276565 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1743bf-9209-4bba-a5dd-82ffaee999f3 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:20.290404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 523/1070] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:16:20 2022] PUT /identity/v3/projects/32bbb936501846e8914acf07890c2891/users/f5fa28ffdafd435cb2d93bd453add3cb/roles/9940e26019d341f4a9af2fa399a02bc8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:16:20.293690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad2ab8-e031-4458-977f-7f08b6570394 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:20.307470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:20.335720 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6def1d0-cca4-4ca9-9a8a-5ba3c7dbad8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:20.336366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6def1d0-cca4-4ca9-9a8a-5ba3c7dbad8b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:20.336657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6def1d0-cca4-4ca9-9a8a-5ba3c7dbad8b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:20.336934 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6def1d0-cca4-4ca9-9a8a-5ba3c7dbad8b None admin] PATH_INFO: `/v3/projects/bcd5a727c1a249e387c437a91818a7d1` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:20.338085 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6def1d0-cca4-4ca9-9a8a-5ba3c7dbad8b None admin] RBAC: Authorizing `identity:get_project(project_id=bcd5a727c1a249e387c437a91818a7d1)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:20.339542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6def1d0-cca4-4ca9-9a8a-5ba3c7dbad8b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:20.341432 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 548/1071] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:16:20 2022] GET /identity/v3/projects/bcd5a727c1a249e387c437a91818a7d1 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:20.351388 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1743bf-9209-4bba-a5dd-82ffaee999f3 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:20.395964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:20.465152 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1743bf-9209-4bba-a5dd-82ffaee999f3 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:20.504116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e207f0-a20d-4443-893c-459301b30182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:20.504730 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e207f0-a20d-4443-893c-459301b30182 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:20.505010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e207f0-a20d-4443-893c-459301b30182 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:20.505848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e207f0-a20d-4443-893c-459301b30182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:20.506578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e207f0-a20d-4443-893c-459301b30182 None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=bcd5a727c1a249e387c437a91818a7d1, user.id=12b3998d0e044b8cba6f29bd8e9bcff7) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:20.506885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e207f0-a20d-4443-893c-459301b30182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:20.508361 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e207f0-a20d-4443-893c-459301b30182 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:20.519653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 524/1072] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:16:20 2022] GET /identity/v3/role_assignments?user.id=12b3998d0e044b8cba6f29bd8e9bcff7&scope.project.id=bcd5a727c1a249e387c437a91818a7d1&role.id=23538dc4c770454e80dd11ed8e498539 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:20.586710 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aa29d-1b94-4f39-8bd4-2774748dcf30 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:20.587280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aa29d-1b94-4f39-8bd4-2774748dcf30 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:20.587280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aa29d-1b94-4f39-8bd4-2774748dcf30 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:20.588644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 549/1073] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:20.594561 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d29b8-b2aa-4cd5-bbbf-7f7d683cc787 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:20.594909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d29b8-b2aa-4cd5-bbbf-7f7d683cc787 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:20.595186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d29b8-b2aa-4cd5-bbbf-7f7d683cc787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:20.615670 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-079d29b8-b2aa-4cd5-bbbf-7f7d683cc787 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:20.616886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-079d29b8-b2aa-4cd5-bbbf-7f7d683cc787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:20.693559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 525/1074] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:20.960078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510aa29d-1b94-4f39-8bd4-2774748dcf30 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:21.061943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510aa29d-1b94-4f39-8bd4-2774748dcf30 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:21.095944 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fadddd-43a0-4266-94bc-1f839e9b13f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:21.096553 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fadddd-43a0-4266-94bc-1f839e9b13f8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:21.096832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fadddd-43a0-4266-94bc-1f839e9b13f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:21.097108 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fadddd-43a0-4266-94bc-1f839e9b13f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:21.141719 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:21.164642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fadddd-43a0-4266-94bc-1f839e9b13f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:21.166098 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fadddd-43a0-4266-94bc-1f839e9b13f8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:21.266290 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 550/1075] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:16:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:16:21.272036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d29b8-b2aa-4cd5-bbbf-7f7d683cc787 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:21.303282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:21.371316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d29b8-b2aa-4cd5-bbbf-7f7d683cc787 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:21.407353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448c080-bab3-4af3-8423-8f2c341d87e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:21.408015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448c080-bab3-4af3-8423-8f2c341d87e9 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:21.408294 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448c080-bab3-4af3-8423-8f2c341d87e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:21.408568 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448c080-bab3-4af3-8423-8f2c341d87e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:21.419053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448c080-bab3-4af3-8423-8f2c341d87e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:21.420518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448c080-bab3-4af3-8423-8f2c341d87e9 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-a448c080-bab3-4af3-8423-8f2c341d87e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:16:21.434632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:21.435668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:16:21.441721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:16:21.441721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:21.441721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:21.441721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 526/1076] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:21.441988 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fadddd-43a0-4266-94bc-1f839e9b13f8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:21.521393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:21.545510 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fadddd-43a0-4266-94bc-1f839e9b13f8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:21.582688 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6355928-fdc0-4c22-82d6-14e4003e9c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:21.583350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6355928-fdc0-4c22-82d6-14e4003e9c32 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:21.583661 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6355928-fdc0-4c22-82d6-14e4003e9c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:21.583939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6355928-fdc0-4c22-82d6-14e4003e9c32 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:21.584575 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6355928-fdc0-4c22-82d6-14e4003e9c32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:21.584882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6355928-fdc0-4c22-82d6-14e4003e9c32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:21.586244 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6355928-fdc0-4c22-82d6-14e4003e9c32 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:21.603598 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 551/1077] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:22.439466 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726912f7-b646-48a9-a2b2-663904eb1edc None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:22.440017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726912f7-b646-48a9-a2b2-663904eb1edc None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:22.440131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726912f7-b646-48a9-a2b2-663904eb1edc None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:22.441397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 527/1078] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:22.447120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab5b5d-1cf3-403e-bb6e-ac865c901948 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:22.447407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab5b5d-1cf3-403e-bb6e-ac865c901948 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:22.447719 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab5b5d-1cf3-403e-bb6e-ac865c901948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:22.465442 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ecab5b5d-1cf3-403e-bb6e-ac865c901948 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:22.466612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ecab5b5d-1cf3-403e-bb6e-ac865c901948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:22.533158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 552/1079] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:22.653841 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dbc08-b66b-401b-b05b-7d47a7aadcab None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:22.654118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dbc08-b66b-401b-b05b-7d47a7aadcab None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:22.654391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dbc08-b66b-401b-b05b-7d47a7aadcab None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:22.655799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 528/1080] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:16:22.661152 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3500a9-191f-4dec-b580-cc36ba7bff22 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:22.661626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3500a9-191f-4dec-b580-cc36ba7bff22 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:22.661721 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3500a9-191f-4dec-b580-cc36ba7bff22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:22.679863 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bf3500a9-191f-4dec-b580-cc36ba7bff22 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:22.681033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bf3500a9-191f-4dec-b580-cc36ba7bff22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:22.740283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:22.745607 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 553/1081] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:22.804010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390dbc08-b66b-401b-b05b-7d47a7aadcab None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:22.886190 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:22.903190 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390dbc08-b66b-401b-b05b-7d47a7aadcab None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:22.940715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7c570-453a-4f7d-9499-0f04a20d2514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:22.941342 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7c570-453a-4f7d-9499-0f04a20d2514 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:22.941622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7c570-453a-4f7d-9499-0f04a20d2514 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:22.941896 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7c570-453a-4f7d-9499-0f04a20d2514 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:22.952623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7c570-453a-4f7d-9499-0f04a20d2514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-efa7c570-453a-4f7d-9499-0f04a20d2514 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:16:22.957446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:22.958558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:22.959453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:22.959453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:22.959453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:22.959453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:22.959453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:16:22.959453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:16:22.959453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:16:22.959453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:16:22.959453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:16:22.959453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:16:22.959453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:16:22.959453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:22.959453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 529/1082] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Sep 12 23:16:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:22.964432 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3500a9-191f-4dec-b580-cc36ba7bff22 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:23.022406 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7c570-453a-4f7d-9499-0f04a20d2514 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:23.066024 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3500a9-191f-4dec-b580-cc36ba7bff22 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:23.101016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134971d2-7f37-4651-886f-e9123eff43db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:23.101620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134971d2-7f37-4651-886f-e9123eff43db None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:23.101925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134971d2-7f37-4651-886f-e9123eff43db None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:23.102198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134971d2-7f37-4651-886f-e9123eff43db None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:23.102852 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134971d2-7f37-4651-886f-e9123eff43db None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:23.103152 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134971d2-7f37-4651-886f-e9123eff43db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:23.104598 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134971d2-7f37-4651-886f-e9123eff43db None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:23.112397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 554/1083] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Sep 12 23:16:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:23.118662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134971d2-7f37-4651-886f-e9123eff43db None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:23.128802 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7c570-453a-4f7d-9499-0f04a20d2514 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:23.161005 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ae3be-3f0b-4f75-bc6b-f6d31843e0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:23.162129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ae3be-3f0b-4f75-bc6b-f6d31843e0cb None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:23.162402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ae3be-3f0b-4f75-bc6b-f6d31843e0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:23.162668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ae3be-3f0b-4f75-bc6b-f6d31843e0cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:23.172220 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ae3be-3f0b-4f75-bc6b-f6d31843e0cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:16:23.176517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-111ae3be-3f0b-4f75-bc6b-f6d31843e0cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:16:23.177493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:23.179126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:23.180001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:23.180001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:23.180001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:23.180001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:23.180001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:16:23.180001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:16:23.180001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:16:23.180001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:16:23.180001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:16:23.180001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:16:23.180001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:16:23.180001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:23.184092 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 530/1084] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:16:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:23.184922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ae3be-3f0b-4f75-bc6b-f6d31843e0cb None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:23.188883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:23.218140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134971d2-7f37-4651-886f-e9123eff43db None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:23.251279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19c65b-438d-4f41-95e3-8b4725815937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:23.251937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19c65b-438d-4f41-95e3-8b4725815937 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:23.252215 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19c65b-438d-4f41-95e3-8b4725815937 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:23.252486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19c65b-438d-4f41-95e3-8b4725815937 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:23.263361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19c65b-438d-4f41-95e3-8b4725815937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:23.264816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19c65b-438d-4f41-95e3-8b4725815937 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ed19c65b-438d-4f41-95e3-8b4725815937 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:16:23.279109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:23.280142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:16:23.281057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:16:23.281057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:23.281057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:23.281057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 555/1085] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:23.284462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ae3be-3f0b-4f75-bc6b-f6d31843e0cb None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:23.286158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19c65b-438d-4f41-95e3-8b4725815937 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:23.318110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b48441-590b-4bc0-ae50-8b3b345677fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:23.318711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b48441-590b-4bc0-ae50-8b3b345677fc None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:23.318991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b48441-590b-4bc0-ae50-8b3b345677fc None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:23.319265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b48441-590b-4bc0-ae50-8b3b345677fc None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:23.319953 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b48441-590b-4bc0-ae50-8b3b345677fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:23.320253 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b48441-590b-4bc0-ae50-8b3b345677fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:23.321624 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b48441-590b-4bc0-ae50-8b3b345677fc None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:23.331181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 531/1086] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:16:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:23.337521 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b48441-590b-4bc0-ae50-8b3b345677fc None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:23.388352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19c65b-438d-4f41-95e3-8b4725815937 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:23.400705 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:23.401486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:23.423004 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5c220-8c4a-4f48-85ba-4a945b7600b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:23.423624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5c220-8c4a-4f48-85ba-4a945b7600b1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:23.423907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5c220-8c4a-4f48-85ba-4a945b7600b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:23.424181 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5c220-8c4a-4f48-85ba-4a945b7600b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:23.424815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5c220-8c4a-4f48-85ba-4a945b7600b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:23.425115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5c220-8c4a-4f48-85ba-4a945b7600b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:23.426471 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5c220-8c4a-4f48-85ba-4a945b7600b1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:23.435372 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b48441-590b-4bc0-ae50-8b3b345677fc None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:23.438571 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 556/1087] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:23.444706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5c220-8c4a-4f48-85ba-4a945b7600b1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:23.469520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cb449-9ace-49dc-9946-4648608d8963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:23.470143 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cb449-9ace-49dc-9946-4648608d8963 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:23.470421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cb449-9ace-49dc-9946-4648608d8963 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:23.470690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cb449-9ace-49dc-9946-4648608d8963 None admin] PATH_INFO: `/v3/users/12b3998d0e044b8cba6f29bd8e9bcff7` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:23.471838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cb449-9ace-49dc-9946-4648608d8963 None admin] RBAC: Authorizing `identity:get_user(user_id=12b3998d0e044b8cba6f29bd8e9bcff7)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:23.473262 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cb449-9ace-49dc-9946-4648608d8963 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:23.474834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 532/1088] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:16:23 2022] GET /identity/v3/users/12b3998d0e044b8cba6f29bd8e9bcff7 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:23.480394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cb449-9ace-49dc-9946-4648608d8963 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:23.544359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5c220-8c4a-4f48-85ba-4a945b7600b1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:23.578639 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cb449-9ace-49dc-9946-4648608d8963 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:23.579473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e7236-7ada-4dc0-ad87-43a5da6bdc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:23.580123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7236-7ada-4dc0-ad87-43a5da6bdc1c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:23.580400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7236-7ada-4dc0-ad87-43a5da6bdc1c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:23.580675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7236-7ada-4dc0-ad87-43a5da6bdc1c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:23.588192 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e7236-7ada-4dc0-ad87-43a5da6bdc1c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:23.589609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e7236-7ada-4dc0-ad87-43a5da6bdc1c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e03e7236-7ada-4dc0-ad87-43a5da6bdc1c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:16:23.601449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:23.602440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:23.603412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:23.603412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:23.603412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:16:23.603412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:16:23.603412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:16:23.603412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:16:23.603412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:16:23.603412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:16:23.603412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:16:23.603412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:23.603412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 557/1089] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Sep 12 23:16:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:16:23.608274 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e7236-7ada-4dc0-ad87-43a5da6bdc1c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:23.613897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d35fe-261e-4f1a-9046-02ff2d58c107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:23.614521 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d35fe-261e-4f1a-9046-02ff2d58c107 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:23.614797 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d35fe-261e-4f1a-9046-02ff2d58c107 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:23.615069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d35fe-261e-4f1a-9046-02ff2d58c107 None admin] PATH_INFO: `/v3/projects/bcd5a727c1a249e387c437a91818a7d1` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:23.616175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d35fe-261e-4f1a-9046-02ff2d58c107 None admin] RBAC: Authorizing `identity:get_project(project_id=bcd5a727c1a249e387c437a91818a7d1)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:23.617590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d35fe-261e-4f1a-9046-02ff2d58c107 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:23.624185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 533/1090] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:16:23 2022] GET /identity/v3/projects/bcd5a727c1a249e387c437a91818a7d1 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:23.625120 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d35fe-261e-4f1a-9046-02ff2d58c107 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:23.627745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:23.639145 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:23.707214 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e7236-7ada-4dc0-ad87-43a5da6bdc1c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:23.722786 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d35fe-261e-4f1a-9046-02ff2d58c107 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:23.740375 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06012d59-919c-4710-9b58-7243dbea988a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:23.740970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06012d59-919c-4710-9b58-7243dbea988a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:23.741248 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06012d59-919c-4710-9b58-7243dbea988a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:23.741522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06012d59-919c-4710-9b58-7243dbea988a None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:23.742236 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06012d59-919c-4710-9b58-7243dbea988a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:23.742543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06012d59-919c-4710-9b58-7243dbea988a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:23.743970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06012d59-919c-4710-9b58-7243dbea988a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:23.756595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704736c-eeb0-4270-9083-a60b66cf5cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:23.757251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704736c-eeb0-4270-9083-a60b66cf5cad None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:23.757534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704736c-eeb0-4270-9083-a60b66cf5cad None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:23.757807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704736c-eeb0-4270-9083-a60b66cf5cad None admin] PATH_INFO: `/v3/projects/bcd5a727c1a249e387c437a91818a7d1/users/12b3998d0e044b8cba6f29bd8e9bcff7/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:23.759742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704736c-eeb0-4270-9083-a60b66cf5cad None admin] RBAC: Authorizing `identity:create_grant(project_id=bcd5a727c1a249e387c437a91818a7d1, user_id=12b3998d0e044b8cba6f29bd8e9bcff7, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:23.760932 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 558/1091] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Sep 12 23:16:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:23.761166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704736c-eeb0-4270-9083-a60b66cf5cad None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:23.766762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06012d59-919c-4710-9b58-7243dbea988a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:23.775096 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 534/1092] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:16:23 2022] PUT /identity/v3/projects/bcd5a727c1a249e387c437a91818a7d1/users/12b3998d0e044b8cba6f29bd8e9bcff7/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:16:23.861334 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:23.871662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06012d59-919c-4710-9b58-7243dbea988a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:23.907420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b86868-b594-4d7f-b92f-17cb3a643f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:23.908077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b86868-b594-4d7f-b92f-17cb3a643f07 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:23.908351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b86868-b594-4d7f-b92f-17cb3a643f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:23.908620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b86868-b594-4d7f-b92f-17cb3a643f07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:23.979058 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b86868-b594-4d7f-b92f-17cb3a643f07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:23.980555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b86868-b594-4d7f-b92f-17cb3a643f07 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:24.065902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:24.081000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 559/1093] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:16:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:16:24.082422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704736c-eeb0-4270-9083-a60b66cf5cad None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:24.142247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:24.182433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704736c-eeb0-4270-9083-a60b66cf5cad None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:24.216212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968c921-89f9-45be-b87a-3d46715c302c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:24.216837 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968c921-89f9-45be-b87a-3d46715c302c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:24.217126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968c921-89f9-45be-b87a-3d46715c302c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:24.217402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968c921-89f9-45be-b87a-3d46715c302c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:24.227931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968c921-89f9-45be-b87a-3d46715c302c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:24.229388 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968c921-89f9-45be-b87a-3d46715c302c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-5968c921-89f9-45be-b87a-3d46715c302c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:16:24.244256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:24.245474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:16:24.246362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:16:24.246362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:16:24.246362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:24.246362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 535/1094] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:24.250986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b86868-b594-4d7f-b92f-17cb3a643f07 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:24.347023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b86868-b594-4d7f-b92f-17cb3a643f07 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:24.379703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82540403-9191-43dd-b978-00978a5bf6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:24.380930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82540403-9191-43dd-b978-00978a5bf6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:24.381210 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82540403-9191-43dd-b978-00978a5bf6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:24.381482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82540403-9191-43dd-b978-00978a5bf6c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:24.382120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82540403-9191-43dd-b978-00978a5bf6c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:24.382424 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82540403-9191-43dd-b978-00978a5bf6c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:24.383800 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82540403-9191-43dd-b978-00978a5bf6c9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:24.399621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 560/1095] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:24.400568 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968c921-89f9-45be-b87a-3d46715c302c None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:24.497409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968c921-89f9-45be-b87a-3d46715c302c None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:24.511223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:24.530931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc0270-58b3-4bb0-9848-c7bc6124a857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:24.531557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc0270-58b3-4bb0-9848-c7bc6124a857 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:24.531863 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc0270-58b3-4bb0-9848-c7bc6124a857 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:24.532405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc0270-58b3-4bb0-9848-c7bc6124a857 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:24.539381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc0270-58b3-4bb0-9848-c7bc6124a857 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:24.540853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc0270-58b3-4bb0-9848-c7bc6124a857 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-b7bc0270-58b3-4bb0-9848-c7bc6124a857 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:16:24.553131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:24.554135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:24.555052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:16:24.555052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:16:24.555052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:16:24.555052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:16:24.555052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:16:24.555052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:16:24.555052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:16:24.555052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:24.555052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 536/1096] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:24.560093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82540403-9191-43dd-b978-00978a5bf6c9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:24.610813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:24.663179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82540403-9191-43dd-b978-00978a5bf6c9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:24.695715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0737ee8-7e8b-4128-ab86-56264e6f7c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:24.696316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737ee8-7e8b-4128-ab86-56264e6f7c51 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:24.696597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737ee8-7e8b-4128-ab86-56264e6f7c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:24.696869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737ee8-7e8b-4128-ab86-56264e6f7c51 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:24.697558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0737ee8-7e8b-4128-ab86-56264e6f7c51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:24.697864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0737ee8-7e8b-4128-ab86-56264e6f7c51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:24.699276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0737ee8-7e8b-4128-ab86-56264e6f7c51 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:24.711353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 561/1097] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:16:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:24.717372 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc0270-58b3-4bb0-9848-c7bc6124a857 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:24.813401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc0270-58b3-4bb0-9848-c7bc6124a857 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:24.844791 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1dce68-a03e-4442-9ed1-13ef84064d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:24.845379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1dce68-a03e-4442-9ed1-13ef84064d5a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:24.845661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1dce68-a03e-4442-9ed1-13ef84064d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:24.845937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1dce68-a03e-4442-9ed1-13ef84064d5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:24.846654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1dce68-a03e-4442-9ed1-13ef84064d5a None admin] RBAC: Adding query filter params (role.id=9940e26019d341f4a9af2fa399a02bc8, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=f5fa28ffdafd435cb2d93bd453add3cb) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:24.846963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1dce68-a03e-4442-9ed1-13ef84064d5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:24.848426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1dce68-a03e-4442-9ed1-13ef84064d5a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:24.854476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 537/1098] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:16:24 2022] GET /identity/v3/role_assignments?user.id=f5fa28ffdafd435cb2d93bd453add3cb&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=9940e26019d341f4a9af2fa399a02bc8 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:25.902092 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1193e5b-fd51-4845-94bf-1f45a782a31e None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:25.902651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1193e5b-fd51-4845-94bf-1f45a782a31e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:25.902651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1193e5b-fd51-4845-94bf-1f45a782a31e None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:25.903985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 562/1099] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:25.909499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26eb36-6ae2-4021-90be-cb30ec8377cb None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:25.909931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26eb36-6ae2-4021-90be-cb30ec8377cb None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:25.910032 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26eb36-6ae2-4021-90be-cb30ec8377cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:25.928374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-1f26eb36-6ae2-4021-90be-cb30ec8377cb None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:25.929546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1f26eb36-6ae2-4021-90be-cb30ec8377cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:25.958106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:25.994063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 538/1100] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:16:26.267759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1193e5b-fd51-4845-94bf-1f45a782a31e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:26.366401 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1193e5b-fd51-4845-94bf-1f45a782a31e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:26.376621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:26.400648 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15194de-69fe-4450-8d80-8a188c152026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:26.401274 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15194de-69fe-4450-8d80-8a188c152026 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:26.401556 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15194de-69fe-4450-8d80-8a188c152026 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:26.401833 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15194de-69fe-4450-8d80-8a188c152026 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:26.411853 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15194de-69fe-4450-8d80-8a188c152026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a15194de-69fe-4450-8d80-8a188c152026 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:16:26.416682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:26.417791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:26.418968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:26.418968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:26.418968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:26.418968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:26.418968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:16:26.418968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:16:26.418968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:16:26.418968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:16:26.418968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:16:26.418968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:16:26.418968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:16:26.418968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:26.418968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 563/1101] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:16:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:26.423799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26eb36-6ae2-4021-90be-cb30ec8377cb None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:26.518765 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26eb36-6ae2-4021-90be-cb30ec8377cb None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:26.550694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88489b-82fa-4665-b32b-4567af181334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:26.551284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88489b-82fa-4665-b32b-4567af181334 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:26.551563 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88489b-82fa-4665-b32b-4567af181334 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:26.552400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88489b-82fa-4665-b32b-4567af181334 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:26.553053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88489b-82fa-4665-b32b-4567af181334 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:26.553354 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88489b-82fa-4665-b32b-4567af181334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:26.554716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88489b-82fa-4665-b32b-4567af181334 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:26.562330 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 539/1102] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:16:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:26.568658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15194de-69fe-4450-8d80-8a188c152026 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:26.667128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15194de-69fe-4450-8d80-8a188c152026 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:26.701827 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258a3f3-9c5b-458c-9a23-f9ad6a6b76c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:26.702648 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258a3f3-9c5b-458c-9a23-f9ad6a6b76c3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:26.703014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258a3f3-9c5b-458c-9a23-f9ad6a6b76c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:26.703376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258a3f3-9c5b-458c-9a23-f9ad6a6b76c3 None admin] PATH_INFO: `/v3/users/12b3998d0e044b8cba6f29bd8e9bcff7` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:26.704697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258a3f3-9c5b-458c-9a23-f9ad6a6b76c3 None admin] RBAC: Authorizing `identity:get_user(user_id=12b3998d0e044b8cba6f29bd8e9bcff7)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:26.706110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258a3f3-9c5b-458c-9a23-f9ad6a6b76c3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:26.707689 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 564/1103] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:16:26 2022] GET /identity/v3/users/12b3998d0e044b8cba6f29bd8e9bcff7 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:26.713489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88489b-82fa-4665-b32b-4567af181334 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:26.743389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:26.811286 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88489b-82fa-4665-b32b-4567af181334 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:26.842647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1cf9c-b4c1-434a-a89e-99651d0f8417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:26.843240 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1cf9c-b4c1-434a-a89e-99651d0f8417 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:26.843522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1cf9c-b4c1-434a-a89e-99651d0f8417 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:26.843820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1cf9c-b4c1-434a-a89e-99651d0f8417 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:26.907674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1cf9c-b4c1-434a-a89e-99651d0f8417 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:26.909113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1cf9c-b4c1-434a-a89e-99651d0f8417 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:26.937698 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:27.004949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 540/1104] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:16:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:16:27.010837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258a3f3-9c5b-458c-9a23-f9ad6a6b76c3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:27.049058 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:27.107271 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258a3f3-9c5b-458c-9a23-f9ad6a6b76c3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:27.139987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf9228-946d-4962-8e6e-3662e8936ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:27.140606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf9228-946d-4962-8e6e-3662e8936ecc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:27.140891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf9228-946d-4962-8e6e-3662e8936ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:27.141170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf9228-946d-4962-8e6e-3662e8936ecc None admin] PATH_INFO: `/v3/projects/bcd5a727c1a249e387c437a91818a7d1` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:27.142277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf9228-946d-4962-8e6e-3662e8936ecc None admin] RBAC: Authorizing `identity:get_project(project_id=bcd5a727c1a249e387c437a91818a7d1)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:27.143727 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf9228-946d-4962-8e6e-3662e8936ecc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:27.145347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 565/1105] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:16:27 2022] GET /identity/v3/projects/bcd5a727c1a249e387c437a91818a7d1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:27.151903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1cf9c-b4c1-434a-a89e-99651d0f8417 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:27.247509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1cf9c-b4c1-434a-a89e-99651d0f8417 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:27.274916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:27.280991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c8802-4bd9-43fc-afdb-82776d924aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:27.281581 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c8802-4bd9-43fc-afdb-82776d924aad None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:27.281861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c8802-4bd9-43fc-afdb-82776d924aad None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:27.282388 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c8802-4bd9-43fc-afdb-82776d924aad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:27.283099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c8802-4bd9-43fc-afdb-82776d924aad None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=bcd5a727c1a249e387c437a91818a7d1, user.id=12b3998d0e044b8cba6f29bd8e9bcff7) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:27.283407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c8802-4bd9-43fc-afdb-82776d924aad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:27.284854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c8802-4bd9-43fc-afdb-82776d924aad None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:27.290423 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 541/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:16:27 2022] GET /identity/v3/role_assignments?user.id=12b3998d0e044b8cba6f29bd8e9bcff7&scope.project.id=bcd5a727c1a249e387c437a91818a7d1&role.id=23538dc4c770454e80dd11ed8e498539 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:29.435162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85ea7f-003c-4ba8-8ba2-12dbec352ce2 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:29.435705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85ea7f-003c-4ba8-8ba2-12dbec352ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:29.435799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85ea7f-003c-4ba8-8ba2-12dbec352ce2 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:29.437056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 566/1107] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:29.443085 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb419-d021-4693-9c70-f9c345d95be9 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:29.443496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb419-d021-4693-9c70-f9c345d95be9 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:29.443669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb419-d021-4693-9c70-f9c345d95be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:29.461259 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-238cb419-d021-4693-9c70-f9c345d95be9 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:29.462440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-238cb419-d021-4693-9c70-f9c345d95be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:29.525992 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 542/1108] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:29.805870 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85ea7f-003c-4ba8-8ba2-12dbec352ce2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:29.913897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85ea7f-003c-4ba8-8ba2-12dbec352ce2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:29.921998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:29.948695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906efd3-9706-411a-ada1-3ece0458cdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:29.949314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906efd3-9706-411a-ada1-3ece0458cdc6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:29.949591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906efd3-9706-411a-ada1-3ece0458cdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:29.949858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906efd3-9706-411a-ada1-3ece0458cdc6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:29.950950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906efd3-9706-411a-ada1-3ece0458cdc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:29.952380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906efd3-9706-411a-ada1-3ece0458cdc6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:29.953923 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 567/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:29.960330 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cb419-d021-4693-9c70-f9c345d95be9 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:30.056695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cb419-d021-4693-9c70-f9c345d95be9 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:30.079692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:30.090851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686d836-9812-4521-8ea5-ec3cfc200c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:30.091467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d836-9812-4521-8ea5-ec3cfc200c55 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:30.091792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d836-9812-4521-8ea5-ec3cfc200c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:30.092067 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d836-9812-4521-8ea5-ec3cfc200c55 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:30.092723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686d836-9812-4521-8ea5-ec3cfc200c55 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:30.094177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686d836-9812-4521-8ea5-ec3cfc200c55 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:30.121738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 543/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:16:29 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:16:32.321131 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fad5f-113d-4e9c-8647-d0a80e6b8435 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:32.321625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fad5f-113d-4e9c-8647-d0a80e6b8435 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:32.321625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fad5f-113d-4e9c-8647-d0a80e6b8435 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:32.322926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 568/1111] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:32.328269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4af6c-85ca-4a7b-a117-fe72f7fcb95b None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:32.328643 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4af6c-85ca-4a7b-a117-fe72f7fcb95b None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:32.328824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4af6c-85ca-4a7b-a117-fe72f7fcb95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:32.347081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-42e4af6c-85ca-4a7b-a117-fe72f7fcb95b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:32.348298 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-42e4af6c-85ca-4a7b-a117-fe72f7fcb95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:32.414439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 544/1112] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:32.688464 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fad5f-113d-4e9c-8647-d0a80e6b8435 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:32.786855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fad5f-113d-4e9c-8647-d0a80e6b8435 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:32.818685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f69ca-d28d-4ede-82f1-78c535ce5226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:32.819295 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f69ca-d28d-4ede-82f1-78c535ce5226 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:32.819595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f69ca-d28d-4ede-82f1-78c535ce5226 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:32.819870 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f69ca-d28d-4ede-82f1-78c535ce5226 None admin] PATH_INFO: `/v3/roles/31058b9633b44d7ca64f103119987fd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:32.821225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f69ca-d28d-4ede-82f1-78c535ce5226 None admin] RBAC: Authorizing `identity:get_role(role_id=31058b9633b44d7ca64f103119987fd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:32.822571 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f69ca-d28d-4ede-82f1-78c535ce5226 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:32.823900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 569/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:16:32 2022] GET /identity/v3/roles/31058b9633b44d7ca64f103119987fd9 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:32.829713 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4af6c-85ca-4a7b-a117-fe72f7fcb95b None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:32.923733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4af6c-85ca-4a7b-a117-fe72f7fcb95b None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:32.934693 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:32.958113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1633d-9f22-4d8e-a6d9-b5de2dc7281d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:32.958673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1633d-9f22-4d8e-a6d9-b5de2dc7281d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:32.958952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1633d-9f22-4d8e-a6d9-b5de2dc7281d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:32.959227 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1633d-9f22-4d8e-a6d9-b5de2dc7281d None admin] PATH_INFO: `/v3/users/90075cca37124d32be0892ef246d2915` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:32.971812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1633d-9f22-4d8e-a6d9-b5de2dc7281d None admin] RBAC: Authorizing `identity:get_user(user_id=90075cca37124d32be0892ef246d2915)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:32.973769 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1633d-9f22-4d8e-a6d9-b5de2dc7281d None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:32.975358 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 545/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:16:32 2022] GET /identity/v3/users/90075cca37124d32be0892ef246d2915 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:32.981219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f69ca-d28d-4ede-82f1-78c535ce5226 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:33.022400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:33.076257 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f69ca-d28d-4ede-82f1-78c535ce5226 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:33.108175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f8dd9-ea85-4a52-897a-0f4eee22b434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:33.108761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f8dd9-ea85-4a52-897a-0f4eee22b434 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:33.109039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f8dd9-ea85-4a52-897a-0f4eee22b434 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:33.109308 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f8dd9-ea85-4a52-897a-0f4eee22b434 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:33.173030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f8dd9-ea85-4a52-897a-0f4eee22b434 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:33.174445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f8dd9-ea85-4a52-897a-0f4eee22b434 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:33.213168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:33.267721 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 570/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:16:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:16:33.273701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1633d-9f22-4d8e-a6d9-b5de2dc7281d None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:33.368691 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1633d-9f22-4d8e-a6d9-b5de2dc7281d None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:33.401943 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c87e9-ee41-4811-a8b5-94b20f8ece04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:33.402550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c87e9-ee41-4811-a8b5-94b20f8ece04 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:33.402830 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c87e9-ee41-4811-a8b5-94b20f8ece04 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:33.403101 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c87e9-ee41-4811-a8b5-94b20f8ece04 None admin] PATH_INFO: `/v3/projects/bcd5a727c1a249e387c437a91818a7d1` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:33.404214 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c87e9-ee41-4811-a8b5-94b20f8ece04 None admin] RBAC: Authorizing `identity:get_project(project_id=bcd5a727c1a249e387c437a91818a7d1)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:33.405636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c87e9-ee41-4811-a8b5-94b20f8ece04 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:33.407472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 546/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:16:33 2022] GET /identity/v3/projects/bcd5a727c1a249e387c437a91818a7d1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:33.413532 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f8dd9-ea85-4a52-897a-0f4eee22b434 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:33.507321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f8dd9-ea85-4a52-897a-0f4eee22b434 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:33.540263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695b998-9002-4131-9e57-a84babb6d737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:33.540849 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695b998-9002-4131-9e57-a84babb6d737 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:33.541127 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695b998-9002-4131-9e57-a84babb6d737 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:33.541398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695b998-9002-4131-9e57-a84babb6d737 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:33.542100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695b998-9002-4131-9e57-a84babb6d737 None admin] RBAC: Adding query filter params (role.id=31058b9633b44d7ca64f103119987fd9, scope.project.id=bcd5a727c1a249e387c437a91818a7d1, user.id=90075cca37124d32be0892ef246d2915) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:33.542405 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695b998-9002-4131-9e57-a84babb6d737 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:33.543832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695b998-9002-4131-9e57-a84babb6d737 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:33.548903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 571/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:16:33 2022] GET /identity/v3/role_assignments?user.id=90075cca37124d32be0892ef246d2915&scope.project.id=bcd5a727c1a249e387c437a91818a7d1&role.id=31058b9633b44d7ca64f103119987fd9 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:35.636819 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fae4c-0705-4834-8578-5ec1006df22e None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:35.637376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fae4c-0705-4834-8578-5ec1006df22e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:35.637376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fae4c-0705-4834-8578-5ec1006df22e None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:35.638700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 547/1118] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:35.643912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58f22f-5429-4dcf-acde-840d61cc6bf5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:35.644299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58f22f-5429-4dcf-acde-840d61cc6bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:35.644472 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58f22f-5429-4dcf-acde-840d61cc6bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:35.657912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:35.664261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0f58f22f-5429-4dcf-acde-840d61cc6bf5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:35.665448 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0f58f22f-5429-4dcf-acde-840d61cc6bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:35.729614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 572/1119] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:35.996390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148fae4c-0705-4834-8578-5ec1006df22e None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:36.031478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:36.093446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148fae4c-0705-4834-8578-5ec1006df22e None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:36.125689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf784d0-5b10-4631-ae8e-ad42abcb2da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:36.126297 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf784d0-5b10-4631-ae8e-ad42abcb2da5 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:36.126576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf784d0-5b10-4631-ae8e-ad42abcb2da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:36.126847 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf784d0-5b10-4631-ae8e-ad42abcb2da5 None admin] PATH_INFO: `/v3/roles/31058b9633b44d7ca64f103119987fd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:36.128354 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf784d0-5b10-4631-ae8e-ad42abcb2da5 None admin] RBAC: Authorizing `identity:get_role(role_id=31058b9633b44d7ca64f103119987fd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:36.130256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf784d0-5b10-4631-ae8e-ad42abcb2da5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:36.131560 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 548/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:16:35 2022] GET /identity/v3/roles/31058b9633b44d7ca64f103119987fd9 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:36.137249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58f22f-5429-4dcf-acde-840d61cc6bf5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:36.232015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58f22f-5429-4dcf-acde-840d61cc6bf5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:36.264976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b42ff-4894-444d-ac03-804b712957f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:36.265591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b42ff-4894-444d-ac03-804b712957f6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:36.265872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b42ff-4894-444d-ac03-804b712957f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:36.266147 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b42ff-4894-444d-ac03-804b712957f6 None admin] PATH_INFO: `/v3/users/90075cca37124d32be0892ef246d2915` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:36.267253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b42ff-4894-444d-ac03-804b712957f6 None admin] RBAC: Authorizing `identity:get_user(user_id=90075cca37124d32be0892ef246d2915)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:36.268700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b42ff-4894-444d-ac03-804b712957f6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:36.270748 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 573/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:16:36 2022] GET /identity/v3/users/90075cca37124d32be0892ef246d2915 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:36.276328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf784d0-5b10-4631-ae8e-ad42abcb2da5 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:36.370579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf784d0-5b10-4631-ae8e-ad42abcb2da5 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:36.372855 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:36.403646 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448b6c3-ba1d-4003-846f-f2cdf76f00d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:36.404260 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448b6c3-ba1d-4003-846f-f2cdf76f00d0 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:36.404540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448b6c3-ba1d-4003-846f-f2cdf76f00d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:36.404811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448b6c3-ba1d-4003-846f-f2cdf76f00d0 None admin] PATH_INFO: `/v3/projects/bcd5a727c1a249e387c437a91818a7d1` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:36.405911 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448b6c3-ba1d-4003-846f-f2cdf76f00d0 None admin] RBAC: Authorizing `identity:get_project(project_id=bcd5a727c1a249e387c437a91818a7d1)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:36.407616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448b6c3-ba1d-4003-846f-f2cdf76f00d0 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:36.409221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 549/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:16:36 2022] GET /identity/v3/projects/bcd5a727c1a249e387c437a91818a7d1 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:36.415700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b42ff-4894-444d-ac03-804b712957f6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:36.419361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:36.510647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b42ff-4894-444d-ac03-804b712957f6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:36.543376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d188ec7-8d99-4be1-b95d-5800b57a8713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:36.544043 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d188ec7-8d99-4be1-b95d-5800b57a8713 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:36.544323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d188ec7-8d99-4be1-b95d-5800b57a8713 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:36.544595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d188ec7-8d99-4be1-b95d-5800b57a8713 None admin] PATH_INFO: `/v3/projects/bcd5a727c1a249e387c437a91818a7d1/users/90075cca37124d32be0892ef246d2915/roles/31058b9633b44d7ca64f103119987fd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:36.546684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d188ec7-8d99-4be1-b95d-5800b57a8713 None admin] RBAC: Authorizing `identity:create_grant(project_id=bcd5a727c1a249e387c437a91818a7d1, user_id=90075cca37124d32be0892ef246d2915, role_id=31058b9633b44d7ca64f103119987fd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:36.548120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d188ec7-8d99-4be1-b95d-5800b57a8713 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:36.560234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 574/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:16:36 2022] PUT /identity/v3/projects/bcd5a727c1a249e387c437a91818a7d1/users/90075cca37124d32be0892ef246d2915/roles/31058b9633b44d7ca64f103119987fd9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:16:38.683300 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b08b13-5f5d-47fa-a2ac-fa993c85d974 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:38.683856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b08b13-5f5d-47fa-a2ac-fa993c85d974 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:38.684005 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b08b13-5f5d-47fa-a2ac-fa993c85d974 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:38.685347 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 550/1124] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:38.690659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7148102-5fa3-43a8-ac38-9cb1a9db99a1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:38.691128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7148102-5fa3-43a8-ac38-9cb1a9db99a1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:38.691226 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7148102-5fa3-43a8-ac38-9cb1a9db99a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:38.709466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c7148102-5fa3-43a8-ac38-9cb1a9db99a1 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:38.710652 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c7148102-5fa3-43a8-ac38-9cb1a9db99a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:38.767578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:38.776601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 575/1125] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:39.053217 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b08b13-5f5d-47fa-a2ac-fa993c85d974 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:39.150656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b08b13-5f5d-47fa-a2ac-fa993c85d974 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:39.182748 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554e2f7-c3f5-4725-b467-f0786fdd95d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:39.183353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554e2f7-c3f5-4725-b467-f0786fdd95d2 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:39.183653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554e2f7-c3f5-4725-b467-f0786fdd95d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:39.183924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554e2f7-c3f5-4725-b467-f0786fdd95d2 None admin] PATH_INFO: `/v3/roles/31058b9633b44d7ca64f103119987fd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:39.185281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554e2f7-c3f5-4725-b467-f0786fdd95d2 None admin] RBAC: Authorizing `identity:get_role(role_id=31058b9633b44d7ca64f103119987fd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:39.186633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554e2f7-c3f5-4725-b467-f0786fdd95d2 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:39.187946 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 551/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:16:39 2022] GET /identity/v3/roles/31058b9633b44d7ca64f103119987fd9 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:39.194300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7148102-5fa3-43a8-ac38-9cb1a9db99a1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:39.286713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7148102-5fa3-43a8-ac38-9cb1a9db99a1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:39.319271 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb44e40-8124-46b4-9d3b-ac7058c4f416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:39.319862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb44e40-8124-46b4-9d3b-ac7058c4f416 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:39.320140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb44e40-8124-46b4-9d3b-ac7058c4f416 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:39.320409 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb44e40-8124-46b4-9d3b-ac7058c4f416 None admin] PATH_INFO: `/v3/users/90075cca37124d32be0892ef246d2915` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:39.321502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb44e40-8124-46b4-9d3b-ac7058c4f416 None admin] RBAC: Authorizing `identity:get_user(user_id=90075cca37124d32be0892ef246d2915)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:39.322902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb44e40-8124-46b4-9d3b-ac7058c4f416 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:39.324450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 576/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:16:39 2022] GET /identity/v3/users/90075cca37124d32be0892ef246d2915 => generated 286 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:39.329940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554e2f7-c3f5-4725-b467-f0786fdd95d2 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:39.369578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:39.424915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554e2f7-c3f5-4725-b467-f0786fdd95d2 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:39.457543 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b43b5-fe26-42bb-9fcf-118488bfefbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:39.458132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b43b5-fe26-42bb-9fcf-118488bfefbc None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:39.458410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b43b5-fe26-42bb-9fcf-118488bfefbc None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:39.458683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b43b5-fe26-42bb-9fcf-118488bfefbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:39.526824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b43b5-fe26-42bb-9fcf-118488bfefbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:39.528288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b43b5-fe26-42bb-9fcf-118488bfefbc None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:39.566850 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:39.622968 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 552/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:16:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:16:39.628857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb44e40-8124-46b4-9d3b-ac7058c4f416 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:39.687574 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:39.724427 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb44e40-8124-46b4-9d3b-ac7058c4f416 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:39.756653 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e83a6-f5f9-48a6-bb12-c6a39963af86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:39.757262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e83a6-f5f9-48a6-bb12-c6a39963af86 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:39.757540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e83a6-f5f9-48a6-bb12-c6a39963af86 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:39.757813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e83a6-f5f9-48a6-bb12-c6a39963af86 None admin] PATH_INFO: `/v3/projects/bcd5a727c1a249e387c437a91818a7d1` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:39.758910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e83a6-f5f9-48a6-bb12-c6a39963af86 None admin] RBAC: Authorizing `identity:get_project(project_id=bcd5a727c1a249e387c437a91818a7d1)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:39.760349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e83a6-f5f9-48a6-bb12-c6a39963af86 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:39.761937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 577/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:16:39 2022] GET /identity/v3/projects/bcd5a727c1a249e387c437a91818a7d1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:39.767955 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b43b5-fe26-42bb-9fcf-118488bfefbc None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:39.865111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b43b5-fe26-42bb-9fcf-118488bfefbc None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:39.897410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c575ef-1ef1-4ac0-9f1f-f2567fe304de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:39.897940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c575ef-1ef1-4ac0-9f1f-f2567fe304de None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:39.898218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c575ef-1ef1-4ac0-9f1f-f2567fe304de None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:39.898489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c575ef-1ef1-4ac0-9f1f-f2567fe304de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:39.899199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c575ef-1ef1-4ac0-9f1f-f2567fe304de None admin] RBAC: Adding query filter params (role.id=31058b9633b44d7ca64f103119987fd9, scope.project.id=bcd5a727c1a249e387c437a91818a7d1, user.id=90075cca37124d32be0892ef246d2915) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:39.899504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c575ef-1ef1-4ac0-9f1f-f2567fe304de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:39.900947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c575ef-1ef1-4ac0-9f1f-f2567fe304de None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:39.906417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 553/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:16:39 2022] GET /identity/v3/role_assignments?user.id=90075cca37124d32be0892ef246d2915&scope.project.id=bcd5a727c1a249e387c437a91818a7d1&role.id=31058b9633b44d7ca64f103119987fd9 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:42.068062 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732c9a2-0ac1-4007-84ee-5b481301b26f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:42.068590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732c9a2-0ac1-4007-84ee-5b481301b26f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:42.068590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732c9a2-0ac1-4007-84ee-5b481301b26f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:42.069927 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 578/1131] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:42.075256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311772c8-cdac-402f-852a-f3e34374e9d9 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:42.075699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311772c8-cdac-402f-852a-f3e34374e9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:42.075842 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311772c8-cdac-402f-852a-f3e34374e9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:42.083758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:42.094721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-311772c8-cdac-402f-852a-f3e34374e9d9 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:42.095956 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-311772c8-cdac-402f-852a-f3e34374e9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:42.160388 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 554/1132] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:42.427068 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732c9a2-0ac1-4007-84ee-5b481301b26f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:42.523705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732c9a2-0ac1-4007-84ee-5b481301b26f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:42.550404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:42.556407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56754d83-ac6e-4f7f-84dd-a459282b25eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:42.557013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56754d83-ac6e-4f7f-84dd-a459282b25eb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:42.557291 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56754d83-ac6e-4f7f-84dd-a459282b25eb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:42.557563 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56754d83-ac6e-4f7f-84dd-a459282b25eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:42.558651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56754d83-ac6e-4f7f-84dd-a459282b25eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:42.560099 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56754d83-ac6e-4f7f-84dd-a459282b25eb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:42.561614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 579/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:42.567746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311772c8-cdac-402f-852a-f3e34374e9d9 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:42.661337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311772c8-cdac-402f-852a-f3e34374e9d9 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:42.693865 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a187b79-7005-4456-a492-63bb629ff0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:42.694392 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a187b79-7005-4456-a492-63bb629ff0fc None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:42.694668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a187b79-7005-4456-a492-63bb629ff0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:42.694936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a187b79-7005-4456-a492-63bb629ff0fc None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:42.695595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a187b79-7005-4456-a492-63bb629ff0fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:42.697015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a187b79-7005-4456-a492-63bb629ff0fc None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:42.701351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:42.723103 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 555/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Sep 12 23:16:42 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:16:44.897143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe296dea-cba7-4d4d-a035-6aaf97246f4f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:44.897678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe296dea-cba7-4d4d-a035-6aaf97246f4f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:44.897678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe296dea-cba7-4d4d-a035-6aaf97246f4f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:44.899014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 580/1135] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:44.904273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d1fd2-f43d-4fe6-bdd1-7c5aaa56ded8 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:44.904712 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d1fd2-f43d-4fe6-bdd1-7c5aaa56ded8 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:44.904833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d1fd2-f43d-4fe6-bdd1-7c5aaa56ded8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:44.923173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-722d1fd2-f43d-4fe6-bdd1-7c5aaa56ded8 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:44.924406 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-722d1fd2-f43d-4fe6-bdd1-7c5aaa56ded8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:44.988760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 556/1136] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:45.260686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe296dea-cba7-4d4d-a035-6aaf97246f4f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:45.357165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe296dea-cba7-4d4d-a035-6aaf97246f4f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:45.390664 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec2ced-821b-40df-8a8b-1d9d2ebad247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:45.391269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec2ced-821b-40df-8a8b-1d9d2ebad247 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:45.391546 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec2ced-821b-40df-8a8b-1d9d2ebad247 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:45.391846 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec2ced-821b-40df-8a8b-1d9d2ebad247 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:45.392931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec2ced-821b-40df-8a8b-1d9d2ebad247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:45.394361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec2ced-821b-40df-8a8b-1d9d2ebad247 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:45.395924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 581/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:16:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:45.401486 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d1fd2-f43d-4fe6-bdd1-7c5aaa56ded8 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:45.496302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d1fd2-f43d-4fe6-bdd1-7c5aaa56ded8 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:45.514390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:45.539499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965b1e5-d45f-49f4-b22e-4a20df23477e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:45.540414 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965b1e5-d45f-49f4-b22e-4a20df23477e None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:45.540694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965b1e5-d45f-49f4-b22e-4a20df23477e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:45.540966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965b1e5-d45f-49f4-b22e-4a20df23477e None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:45.541617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965b1e5-d45f-49f4-b22e-4a20df23477e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:45.543058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965b1e5-d45f-49f4-b22e-4a20df23477e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:45.564091 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 557/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:16:45 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:16:47.731678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dbf3a-2767-4d38-84b6-f7b10dc4d064 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:47.732235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dbf3a-2767-4d38-84b6-f7b10dc4d064 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:47.732235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dbf3a-2767-4d38-84b6-f7b10dc4d064 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:47.733536 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 582/1139] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:47.739126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd3e09-2c8e-4c8f-803e-a3e8b74aad03 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:47.739522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd3e09-2c8e-4c8f-803e-a3e8b74aad03 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:47.739691 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd3e09-2c8e-4c8f-803e-a3e8b74aad03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:47.757858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-91fd3e09-2c8e-4c8f-803e-a3e8b74aad03 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:47.759030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-91fd3e09-2c8e-4c8f-803e-a3e8b74aad03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:47.826894 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 558/1140] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:48.097898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9dbf3a-2767-4d38-84b6-f7b10dc4d064 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:48.162850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:48.197018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9dbf3a-2767-4d38-84b6-f7b10dc4d064 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:48.230008 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e895e-65cc-49b6-b4e1-2b024cc9bf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:48.230620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e895e-65cc-49b6-b4e1-2b024cc9bf35 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:48.230904 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e895e-65cc-49b6-b4e1-2b024cc9bf35 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:48.231179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e895e-65cc-49b6-b4e1-2b024cc9bf35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:48.240855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e895e-65cc-49b6-b4e1-2b024cc9bf35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a78e895e-65cc-49b6-b4e1-2b024cc9bf35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:16:48.245656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:48.246752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:48.248053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:48.248053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:48.248053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:48.248053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:48.248053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:16:48.248053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:16:48.248053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:16:48.248053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:16:48.248053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:16:48.248053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:16:48.248053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:16:48.248053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:48.248053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 583/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:16:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:48.252635 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd3e09-2c8e-4c8f-803e-a3e8b74aad03 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:48.347194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd3e09-2c8e-4c8f-803e-a3e8b74aad03 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:48.349302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:48.381880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287da0d-1534-448a-9036-4ef3c452a584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:48.382475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287da0d-1534-448a-9036-4ef3c452a584 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:48.382755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287da0d-1534-448a-9036-4ef3c452a584 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:48.383029 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287da0d-1534-448a-9036-4ef3c452a584 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:48.383716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b287da0d-1534-448a-9036-4ef3c452a584 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:48.384020 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b287da0d-1534-448a-9036-4ef3c452a584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:48.385393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b287da0d-1534-448a-9036-4ef3c452a584 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:48.393137 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 559/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:16:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:48.399396 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e895e-65cc-49b6-b4e1-2b024cc9bf35 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:48.495259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e895e-65cc-49b6-b4e1-2b024cc9bf35 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:48.519774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:48.528431 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd6cfa-2b1b-4b16-ac2e-2bc6e97a211f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:48.529040 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd6cfa-2b1b-4b16-ac2e-2bc6e97a211f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:48.529323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd6cfa-2b1b-4b16-ac2e-2bc6e97a211f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:48.529598 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd6cfa-2b1b-4b16-ac2e-2bc6e97a211f None admin] PATH_INFO: `/v3/users/2f9c1b4f8d594654bd5f2945ddf05373` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:48.542352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd6cfa-2b1b-4b16-ac2e-2bc6e97a211f None admin] RBAC: Authorizing `identity:get_user(user_id=2f9c1b4f8d594654bd5f2945ddf05373)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:48.543800 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd6cfa-2b1b-4b16-ac2e-2bc6e97a211f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:48.545353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 584/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:16:48 2022] GET /identity/v3/users/2f9c1b4f8d594654bd5f2945ddf05373 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:48.550918 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287da0d-1534-448a-9036-4ef3c452a584 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:48.646838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287da0d-1534-448a-9036-4ef3c452a584 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:48.678436 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37f2dd-7048-4cd3-9b02-328bc79b796d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:48.679028 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37f2dd-7048-4cd3-9b02-328bc79b796d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:48.679309 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37f2dd-7048-4cd3-9b02-328bc79b796d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:48.679605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37f2dd-7048-4cd3-9b02-328bc79b796d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:48.708631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:48.743698 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37f2dd-7048-4cd3-9b02-328bc79b796d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:48.745126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37f2dd-7048-4cd3-9b02-328bc79b796d None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:48.837762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 560/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:16:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:16:48.843589 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd6cfa-2b1b-4b16-ac2e-2bc6e97a211f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:48.942006 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd6cfa-2b1b-4b16-ac2e-2bc6e97a211f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:48.974767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5863ef-0e30-4c9f-940f-72ee2dbee384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:48.975392 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5863ef-0e30-4c9f-940f-72ee2dbee384 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:48.975695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5863ef-0e30-4c9f-940f-72ee2dbee384 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:48.975968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5863ef-0e30-4c9f-940f-72ee2dbee384 None admin] PATH_INFO: `/v3/projects/c1bbf1d177214d8f9e4aec3212853d74` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:48.977056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5863ef-0e30-4c9f-940f-72ee2dbee384 None admin] RBAC: Authorizing `identity:get_project(project_id=c1bbf1d177214d8f9e4aec3212853d74)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:48.978476 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5863ef-0e30-4c9f-940f-72ee2dbee384 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:48.980318 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 585/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:16:48 2022] GET /identity/v3/projects/c1bbf1d177214d8f9e4aec3212853d74 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:48.986178 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37f2dd-7048-4cd3-9b02-328bc79b796d None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:49.046379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:49.083242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37f2dd-7048-4cd3-9b02-328bc79b796d None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:49.115250 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d497afd-e92c-47f5-bef3-c543bb4a7888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:49.115863 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d497afd-e92c-47f5-bef3-c543bb4a7888 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:49.116142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d497afd-e92c-47f5-bef3-c543bb4a7888 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:49.116412 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d497afd-e92c-47f5-bef3-c543bb4a7888 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:49.117117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d497afd-e92c-47f5-bef3-c543bb4a7888 None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=c1bbf1d177214d8f9e4aec3212853d74, user.id=2f9c1b4f8d594654bd5f2945ddf05373) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:49.117419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d497afd-e92c-47f5-bef3-c543bb4a7888 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:49.118840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d497afd-e92c-47f5-bef3-c543bb4a7888 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:49.123948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 561/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:16:48 2022] GET /identity/v3/role_assignments?user.id=2f9c1b4f8d594654bd5f2945ddf05373&scope.project.id=c1bbf1d177214d8f9e4aec3212853d74&role.id=23538dc4c770454e80dd11ed8e498539 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:51.241132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31147731-b3c1-448c-b51a-aa29cdd2e208 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:51.241132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31147731-b3c1-448c-b51a-aa29cdd2e208 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:51.241132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31147731-b3c1-448c-b51a-aa29cdd2e208 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:51.242628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 586/1147] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:51.247423 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b8af8-8131-472d-8d8c-bd2a8d1bf679 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:51.247839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b8af8-8131-472d-8d8c-bd2a8d1bf679 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:51.248003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b8af8-8131-472d-8d8c-bd2a8d1bf679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:51.265759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-540b8af8-8131-472d-8d8c-bd2a8d1bf679 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:51.266948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-540b8af8-8131-472d-8d8c-bd2a8d1bf679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:51.332277 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 562/1148] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:51.601050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31147731-b3c1-448c-b51a-aa29cdd2e208 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:51.652683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:51.698332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31147731-b3c1-448c-b51a-aa29cdd2e208 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:51.730486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4ee2f-3b60-460c-9f49-18d7bb7724e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:51.731096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4ee2f-3b60-460c-9f49-18d7bb7724e0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:51.731376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4ee2f-3b60-460c-9f49-18d7bb7724e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:51.731674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4ee2f-3b60-460c-9f49-18d7bb7724e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:51.742173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4ee2f-3b60-460c-9f49-18d7bb7724e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-98b4ee2f-3b60-460c-9f49-18d7bb7724e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:16:51.747031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:51.748241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:51.749341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:51.749341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:51.749341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:51.749341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:51.749341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:16:51.749341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:16:51.749341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:16:51.749341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:16:51.749341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:16:51.749341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:16:51.749341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:16:51.749341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:51.749341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 587/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:16:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:51.754022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b8af8-8131-472d-8d8c-bd2a8d1bf679 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:51.809807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:51.849627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b8af8-8131-472d-8d8c-bd2a8d1bf679 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:51.882554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396a743-f968-4980-baca-9b10c8d08106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:51.883094 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396a743-f968-4980-baca-9b10c8d08106 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:51.883371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396a743-f968-4980-baca-9b10c8d08106 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:51.883670 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396a743-f968-4980-baca-9b10c8d08106 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:51.884322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396a743-f968-4980-baca-9b10c8d08106 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:51.884625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396a743-f968-4980-baca-9b10c8d08106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:51.885986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396a743-f968-4980-baca-9b10c8d08106 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:51.893693 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 563/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:16:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:51.899900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4ee2f-3b60-460c-9f49-18d7bb7724e0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:51.995661 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4ee2f-3b60-460c-9f49-18d7bb7724e0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:52.008225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:52.029372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac568ca-e1a7-4748-838a-8e087ab48c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:52.029992 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac568ca-e1a7-4748-838a-8e087ab48c39 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:52.030269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac568ca-e1a7-4748-838a-8e087ab48c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:52.030542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac568ca-e1a7-4748-838a-8e087ab48c39 None admin] PATH_INFO: `/v3/users/2f9c1b4f8d594654bd5f2945ddf05373` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:52.031924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac568ca-e1a7-4748-838a-8e087ab48c39 None admin] RBAC: Authorizing `identity:get_user(user_id=2f9c1b4f8d594654bd5f2945ddf05373)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:52.033337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac568ca-e1a7-4748-838a-8e087ab48c39 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:52.035484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 588/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:16:51 2022] GET /identity/v3/users/2f9c1b4f8d594654bd5f2945ddf05373 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:16:52.041504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396a743-f968-4980-baca-9b10c8d08106 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:52.137495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396a743-f968-4980-baca-9b10c8d08106 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:52.156534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:52.171046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4fe88-af3c-4b21-a519-eefe3558613c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:52.171695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4fe88-af3c-4b21-a519-eefe3558613c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:52.171980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4fe88-af3c-4b21-a519-eefe3558613c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:52.172257 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4fe88-af3c-4b21-a519-eefe3558613c None admin] PATH_INFO: `/v3/projects/c1bbf1d177214d8f9e4aec3212853d74` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:52.173359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4fe88-af3c-4b21-a519-eefe3558613c None admin] RBAC: Authorizing `identity:get_project(project_id=c1bbf1d177214d8f9e4aec3212853d74)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:52.174793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4fe88-af3c-4b21-a519-eefe3558613c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:52.176483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 564/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:16:52 2022] GET /identity/v3/projects/c1bbf1d177214d8f9e4aec3212853d74 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:52.182261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac568ca-e1a7-4748-838a-8e087ab48c39 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:52.277900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac568ca-e1a7-4748-838a-8e087ab48c39 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:52.310989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c607dd4-24ba-435f-955d-56d4775f1137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:52.311661 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c607dd4-24ba-435f-955d-56d4775f1137 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:52.311944 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c607dd4-24ba-435f-955d-56d4775f1137 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:52.312219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c607dd4-24ba-435f-955d-56d4775f1137 None admin] PATH_INFO: `/v3/projects/c1bbf1d177214d8f9e4aec3212853d74/users/2f9c1b4f8d594654bd5f2945ddf05373/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:52.314090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c607dd4-24ba-435f-955d-56d4775f1137 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1bbf1d177214d8f9e4aec3212853d74, user_id=2f9c1b4f8d594654bd5f2945ddf05373, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:52.315503 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c607dd4-24ba-435f-955d-56d4775f1137 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:52.328978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 589/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:16:52 2022] PUT /identity/v3/projects/c1bbf1d177214d8f9e4aec3212853d74/users/2f9c1b4f8d594654bd5f2945ddf05373/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:16:54.446794 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28db2bf-46c1-42d4-83b2-74906185b892 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:54.447320 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28db2bf-46c1-42d4-83b2-74906185b892 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:54.447320 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28db2bf-46c1-42d4-83b2-74906185b892 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:54.448690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 565/1154] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:54.453792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f62a3-7585-4a19-9056-60b42650dc17 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:54.454154 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f62a3-7585-4a19-9056-60b42650dc17 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:54.454344 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f62a3-7585-4a19-9056-60b42650dc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:54.471954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-517f62a3-7585-4a19-9056-60b42650dc17 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:54.473269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-517f62a3-7585-4a19-9056-60b42650dc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:54.506604 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:54.538850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 590/1155] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:54.808662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28db2bf-46c1-42d4-83b2-74906185b892 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:54.907316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28db2bf-46c1-42d4-83b2-74906185b892 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:54.940594 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d3166-dc41-47ba-a713-7112fdd693ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:54.941844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d3166-dc41-47ba-a713-7112fdd693ba None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:54.942121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d3166-dc41-47ba-a713-7112fdd693ba None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:54.942393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d3166-dc41-47ba-a713-7112fdd693ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:54.952596 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d3166-dc41-47ba-a713-7112fdd693ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-e00d3166-dc41-47ba-a713-7112fdd693ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:16:54.957432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:54.958598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:54.959690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:54.959690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:54.959690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:54.959690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:54.959690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:16:54.959690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:16:54.959690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:16:54.959690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:16:54.959690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:16:54.959690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:16:54.959690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:16:54.959690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:16:54.959690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 566/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:16:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:54.964684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f62a3-7585-4a19-9056-60b42650dc17 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:55.058114 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f62a3-7585-4a19-9056-60b42650dc17 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:55.091121 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3cf0b-39a5-47be-be8f-7dff5d4a2c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:55.091679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3cf0b-39a5-47be-be8f-7dff5d4a2c8b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:55.091963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3cf0b-39a5-47be-be8f-7dff5d4a2c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:55.092237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3cf0b-39a5-47be-be8f-7dff5d4a2c8b None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:55.092893 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3cf0b-39a5-47be-be8f-7dff5d4a2c8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:55.093195 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3cf0b-39a5-47be-be8f-7dff5d4a2c8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:55.094547 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3cf0b-39a5-47be-be8f-7dff5d4a2c8b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:55.102809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 591/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:16:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:55.108636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d3166-dc41-47ba-a713-7112fdd693ba None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:55.160861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:55.212052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d3166-dc41-47ba-a713-7112fdd693ba None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:55.244059 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abbf12-b1c6-4b15-91a0-003301421a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:55.244673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abbf12-b1c6-4b15-91a0-003301421a23 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:55.244954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abbf12-b1c6-4b15-91a0-003301421a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:55.245229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abbf12-b1c6-4b15-91a0-003301421a23 None admin] PATH_INFO: `/v3/users/2f9c1b4f8d594654bd5f2945ddf05373` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:55.246330 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5abbf12-b1c6-4b15-91a0-003301421a23 None admin] RBAC: Authorizing `identity:get_user(user_id=2f9c1b4f8d594654bd5f2945ddf05373)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:55.247781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5abbf12-b1c6-4b15-91a0-003301421a23 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:55.249348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 567/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:16:55 2022] GET /identity/v3/users/2f9c1b4f8d594654bd5f2945ddf05373 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:55.255710 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3cf0b-39a5-47be-be8f-7dff5d4a2c8b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:55.286379 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:55.351794 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3cf0b-39a5-47be-be8f-7dff5d4a2c8b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:55.384059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b46f6-2f47-435b-ae83-956889bd1e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:55.384652 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b46f6-2f47-435b-ae83-956889bd1e34 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:55.384932 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b46f6-2f47-435b-ae83-956889bd1e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:55.385206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b46f6-2f47-435b-ae83-956889bd1e34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:55.450349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b46f6-2f47-435b-ae83-956889bd1e34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:55.451803 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b46f6-2f47-435b-ae83-956889bd1e34 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:55.480042 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:55.546508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 592/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:16:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:16:55.555054 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abbf12-b1c6-4b15-91a0-003301421a23 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:55.651435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abbf12-b1c6-4b15-91a0-003301421a23 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:55.666651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:55.684215 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267a23c-f40c-4343-bb24-3c5b02a04e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:55.684832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267a23c-f40c-4343-bb24-3c5b02a04e51 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:55.685113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267a23c-f40c-4343-bb24-3c5b02a04e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:55.685389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267a23c-f40c-4343-bb24-3c5b02a04e51 None admin] PATH_INFO: `/v3/projects/c1bbf1d177214d8f9e4aec3212853d74` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:55.686486 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267a23c-f40c-4343-bb24-3c5b02a04e51 None admin] RBAC: Authorizing `identity:get_project(project_id=c1bbf1d177214d8f9e4aec3212853d74)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:55.687941 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267a23c-f40c-4343-bb24-3c5b02a04e51 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:55.689550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 568/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:16:55 2022] GET /identity/v3/projects/c1bbf1d177214d8f9e4aec3212853d74 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:55.696202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b46f6-2f47-435b-ae83-956889bd1e34 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:55.788941 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b46f6-2f47-435b-ae83-956889bd1e34 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:55.816589 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:55.821577 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec34e1-7471-46a4-8e89-ff71e9cade32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:55.822162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec34e1-7471-46a4-8e89-ff71e9cade32 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:55.822438 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec34e1-7471-46a4-8e89-ff71e9cade32 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:55.822708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec34e1-7471-46a4-8e89-ff71e9cade32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:55.823412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec34e1-7471-46a4-8e89-ff71e9cade32 None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=c1bbf1d177214d8f9e4aec3212853d74, user.id=2f9c1b4f8d594654bd5f2945ddf05373) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:55.823744 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec34e1-7471-46a4-8e89-ff71e9cade32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:55.825147 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec34e1-7471-46a4-8e89-ff71e9cade32 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:55.830605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 593/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:16:55 2022] GET /identity/v3/role_assignments?user.id=2f9c1b4f8d594654bd5f2945ddf05373&scope.project.id=c1bbf1d177214d8f9e4aec3212853d74&role.id=23538dc4c770454e80dd11ed8e498539 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:16:57.972194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225c85-e9ea-4015-aaec-fbfe9230193c None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:57.972655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225c85-e9ea-4015-aaec-fbfe9230193c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:57.972761 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225c85-e9ea-4015-aaec-fbfe9230193c None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:57.974058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 569/1162] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:16:57.979079 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a908b84-ed8f-4f08-b41d-fb2e4309e674 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:57.979496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a908b84-ed8f-4f08-b41d-fb2e4309e674 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:57.979654 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a908b84-ed8f-4f08-b41d-fb2e4309e674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:57.997859 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9a908b84-ed8f-4f08-b41d-fb2e4309e674 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:16:57.999044 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9a908b84-ed8f-4f08-b41d-fb2e4309e674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:16:58.062943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 594/1163] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:16:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:16:58.336062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13225c85-e9ea-4015-aaec-fbfe9230193c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:58.433055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13225c85-e9ea-4015-aaec-fbfe9230193c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:58.465488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0aff4-7ea5-48c4-86fa-181a3cf48659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:58.466078 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0aff4-7ea5-48c4-86fa-181a3cf48659 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:58.466359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0aff4-7ea5-48c4-86fa-181a3cf48659 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:58.466633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0aff4-7ea5-48c4-86fa-181a3cf48659 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:58.515906 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:58.531279 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0aff4-7ea5-48c4-86fa-181a3cf48659 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:58.533008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0aff4-7ea5-48c4-86fa-181a3cf48659 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:58.625803 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 570/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:16:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:16:58.632064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a908b84-ed8f-4f08-b41d-fb2e4309e674 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:58.725799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a908b84-ed8f-4f08-b41d-fb2e4309e674 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:58.750193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:58.759297 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1e466-b7a2-4e29-8a17-e028e3cf2b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:58.759935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1e466-b7a2-4e29-8a17-e028e3cf2b0a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:58.760217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1e466-b7a2-4e29-8a17-e028e3cf2b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:58.760490 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1e466-b7a2-4e29-8a17-e028e3cf2b0a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:58.766854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1e466-b7a2-4e29-8a17-e028e3cf2b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:58.768307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1e466-b7a2-4e29-8a17-e028e3cf2b0a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-dcf1e466-b7a2-4e29-8a17-e028e3cf2b0a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:16:58.779376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:16:58.780706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:16:58.782126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 595/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Sep 12 23:16:58 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:16:58.786524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0aff4-7ea5-48c4-86fa-181a3cf48659 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:16:58.866401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:16:58.882669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0aff4-7ea5-48c4-86fa-181a3cf48659 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:16:58.915299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3b2b3-a111-4677-aadb-ffdbdab4f9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:16:58.916717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3b2b3-a111-4677-aadb-ffdbdab4f9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:16:58.916717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3b2b3-a111-4677-aadb-ffdbdab4f9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:16:58.916717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3b2b3-a111-4677-aadb-ffdbdab4f9d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:16:58.918099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3b2b3-a111-4677-aadb-ffdbdab4f9d5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:16:58.918401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3b2b3-a111-4677-aadb-ffdbdab4f9d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:16:58.919794 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3b2b3-a111-4677-aadb-ffdbdab4f9d5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:16:58.927604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 571/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Sep 12 23:16:58 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:17:01.032025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b41997-37ac-4d73-a356-0528382980cc None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:01.032579 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b41997-37ac-4d73-a356-0528382980cc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:01.032579 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b41997-37ac-4d73-a356-0528382980cc None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:01.033884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 596/1167] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:17:01.038962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeeaa3-3072-453d-9015-d0bbbfac015c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:01.039398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeeaa3-3072-453d-9015-d0bbbfac015c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:01.039504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeeaa3-3072-453d-9015-d0bbbfac015c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:01.057916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c3aeeaa3-3072-453d-9015-d0bbbfac015c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:17:01.059106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c3aeeaa3-3072-453d-9015-d0bbbfac015c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:17:01.122955 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 572/1168] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:17:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:17:01.389980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b41997-37ac-4d73-a356-0528382980cc None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:01.496970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b41997-37ac-4d73-a356-0528382980cc None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:01.529819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164b9e0-3a74-40f4-bd3f-7565454369ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:01.530434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164b9e0-3a74-40f4-bd3f-7565454369ba None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:01.530719 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164b9e0-3a74-40f4-bd3f-7565454369ba None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:01.530998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164b9e0-3a74-40f4-bd3f-7565454369ba None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:01.531953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164b9e0-3a74-40f4-bd3f-7565454369ba None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:01.533323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164b9e0-3a74-40f4-bd3f-7565454369ba None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:01.548861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 597/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Mon Sep 12 23:17:01 2022] POST /identity/v3/domains => generated 267 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:17:03.728683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d79a4-74fd-42a7-8595-f265a2872a15 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:03.729243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d79a4-74fd-42a7-8595-f265a2872a15 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:03.729243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d79a4-74fd-42a7-8595-f265a2872a15 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:03.730560 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 573/1170] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:17:03.735815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f931e-9d78-43a1-9453-456d82cdcc88 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:03.736196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f931e-9d78-43a1-9453-456d82cdcc88 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:03.736369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f931e-9d78-43a1-9453-456d82cdcc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:03.754505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ad8f931e-9d78-43a1-9453-456d82cdcc88 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:17:03.755703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ad8f931e-9d78-43a1-9453-456d82cdcc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:17:03.795678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:03.820077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 598/1171] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:17:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:17:04.086325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d79a4-74fd-42a7-8595-f265a2872a15 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:04.162228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:04.184153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d79a4-74fd-42a7-8595-f265a2872a15 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:04.216556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573cb47-3fd3-4cb1-9dee-3d68a5dc5969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:04.217168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573cb47-3fd3-4cb1-9dee-3d68a5dc5969 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:04.217445 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573cb47-3fd3-4cb1-9dee-3d68a5dc5969 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:04.217713 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573cb47-3fd3-4cb1-9dee-3d68a5dc5969 None admin] PATH_INFO: `/v3/domains/0ec807500248422da6e8e6edbb3b9f67` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:04.227506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573cb47-3fd3-4cb1-9dee-3d68a5dc5969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0ec807500248422da6e8e6edbb3b9f67)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:04.228965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573cb47-3fd3-4cb1-9dee-3d68a5dc5969 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:04.230523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 574/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Mon Sep 12 23:17:04 2022] GET /identity/v3/domains/0ec807500248422da6e8e6edbb3b9f67 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:17:04.236470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f931e-9d78-43a1-9453-456d82cdcc88 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:04.330811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f931e-9d78-43a1-9453-456d82cdcc88 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:04.362668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c04fb-bf9c-450e-9a12-011a14dda9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:04.363189 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c04fb-bf9c-450e-9a12-011a14dda9ba None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:04.363465 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c04fb-bf9c-450e-9a12-011a14dda9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:04.363760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c04fb-bf9c-450e-9a12-011a14dda9ba None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:04.364402 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c04fb-bf9c-450e-9a12-011a14dda9ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:04.365813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c04fb-bf9c-450e-9a12-011a14dda9ba None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:04.391673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 599/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Sep 12 23:17:04 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:17:06.522428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db953f60-5ce9-43ec-ad66-ee709fd5e094 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:06.522988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db953f60-5ce9-43ec-ad66-ee709fd5e094 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:06.522988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db953f60-5ce9-43ec-ad66-ee709fd5e094 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:06.524384 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 575/1174] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:17:06.529491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1b2a9-0ae2-423f-9487-04bea8e02a26 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:06.529880 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1b2a9-0ae2-423f-9487-04bea8e02a26 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:06.530038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1b2a9-0ae2-423f-9487-04bea8e02a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:06.548520 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d5a1b2a9-0ae2-423f-9487-04bea8e02a26 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:17:06.549725 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d5a1b2a9-0ae2-423f-9487-04bea8e02a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:17:06.571697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:06.614729 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 600/1175] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:17:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:17:06.887179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db953f60-5ce9-43ec-ad66-ee709fd5e094 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:06.987578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db953f60-5ce9-43ec-ad66-ee709fd5e094 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:07.019699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02226373-af70-424d-8493-ad9cb1d5e97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:07.020317 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02226373-af70-424d-8493-ad9cb1d5e97d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:07.020598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02226373-af70-424d-8493-ad9cb1d5e97d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:07.020870 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02226373-af70-424d-8493-ad9cb1d5e97d None admin] PATH_INFO: `/v3/domains/0ec807500248422da6e8e6edbb3b9f67` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:07.021975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02226373-af70-424d-8493-ad9cb1d5e97d None admin] RBAC: Authorizing `identity:get_domain(domain_id=0ec807500248422da6e8e6edbb3b9f67)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:07.023400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02226373-af70-424d-8493-ad9cb1d5e97d None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:07.024991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 576/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Mon Sep 12 23:17:06 2022] GET /identity/v3/domains/0ec807500248422da6e8e6edbb3b9f67 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:17:07.030844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1b2a9-0ae2-423f-9487-04bea8e02a26 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:07.124033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1b2a9-0ae2-423f-9487-04bea8e02a26 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:07.156128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401358c4-5997-419f-953e-cea9e2eedd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:07.156906 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401358c4-5997-419f-953e-cea9e2eedd13 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:07.157181 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401358c4-5997-419f-953e-cea9e2eedd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:07.157447 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401358c4-5997-419f-953e-cea9e2eedd13 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:07.158086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401358c4-5997-419f-953e-cea9e2eedd13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:07.160100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401358c4-5997-419f-953e-cea9e2eedd13 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:07.180628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 601/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:17:07 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:17:09.364625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a36665-672f-458a-8590-bd2e3a0c7334 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:09.365121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a36665-672f-458a-8590-bd2e3a0c7334 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:09.365121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a36665-672f-458a-8590-bd2e3a0c7334 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:09.366430 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 577/1178] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:17:09.371523 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbce1f8-43db-4a95-ab28-1eb2f40e9d1a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:09.371915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbce1f8-43db-4a95-ab28-1eb2f40e9d1a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:09.372102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbce1f8-43db-4a95-ab28-1eb2f40e9d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:09.389639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6cbce1f8-43db-4a95-ab28-1eb2f40e9d1a None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:17:09.390809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6cbce1f8-43db-4a95-ab28-1eb2f40e9d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:17:09.400282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:09.456439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 602/1179] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:17:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:17:09.725270 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a36665-672f-458a-8590-bd2e3a0c7334 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:09.734540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:09.826310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a36665-672f-458a-8590-bd2e3a0c7334 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:09.858106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130e298-03cf-4010-b49e-27b9038794b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:09.858714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130e298-03cf-4010-b49e-27b9038794b4 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:09.858990 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130e298-03cf-4010-b49e-27b9038794b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:09.859260 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130e298-03cf-4010-b49e-27b9038794b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:09.869235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130e298-03cf-4010-b49e-27b9038794b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-b130e298-03cf-4010-b49e-27b9038794b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:17:09.873980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:17:09.875055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:17:09.876065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:17:09.876065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:17:09.876065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:17:09.876065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:17:09.876065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:17:09.876065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:17:09.876065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:17:09.876065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:17:09.876065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:17:09.876065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:17:09.876065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:17:09.876065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:17:09.876065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 578/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:17:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:17:09.881557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbce1f8-43db-4a95-ab28-1eb2f40e9d1a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:09.976307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbce1f8-43db-4a95-ab28-1eb2f40e9d1a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:10.008273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301c7bf-094f-4b60-b494-fb418086bcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:10.008867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301c7bf-094f-4b60-b494-fb418086bcfc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:10.009143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301c7bf-094f-4b60-b494-fb418086bcfc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:10.009411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301c7bf-094f-4b60-b494-fb418086bcfc None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:10.010060 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301c7bf-094f-4b60-b494-fb418086bcfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:17:10.010362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301c7bf-094f-4b60-b494-fb418086bcfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:10.011727 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301c7bf-094f-4b60-b494-fb418086bcfc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:10.019826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 603/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:17:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:17:10.025650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130e298-03cf-4010-b49e-27b9038794b4 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:10.087604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:10.120859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130e298-03cf-4010-b49e-27b9038794b4 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:10.152394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca742c3-c6f3-4d4a-8d7c-7fee494a6161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:10.152999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca742c3-c6f3-4d4a-8d7c-7fee494a6161 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:10.153275 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca742c3-c6f3-4d4a-8d7c-7fee494a6161 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:10.153572 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca742c3-c6f3-4d4a-8d7c-7fee494a6161 None admin] PATH_INFO: `/v3/users/0031ad8c213a4d3ca0f7e9ed9ec0fb34` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:10.166705 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca742c3-c6f3-4d4a-8d7c-7fee494a6161 None admin] RBAC: Authorizing `identity:get_user(user_id=0031ad8c213a4d3ca0f7e9ed9ec0fb34)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:10.168162 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca742c3-c6f3-4d4a-8d7c-7fee494a6161 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:10.169710 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 579/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:17:10 2022] GET /identity/v3/users/0031ad8c213a4d3ca0f7e9ed9ec0fb34 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:17:10.175439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301c7bf-094f-4b60-b494-fb418086bcfc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:10.184268 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:10.271000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301c7bf-094f-4b60-b494-fb418086bcfc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:10.303241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa3d92-56d0-4f78-8731-76b0d3ba681a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:10.303864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa3d92-56d0-4f78-8731-76b0d3ba681a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:10.304144 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa3d92-56d0-4f78-8731-76b0d3ba681a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:10.304415 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa3d92-56d0-4f78-8731-76b0d3ba681a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:10.369264 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa3d92-56d0-4f78-8731-76b0d3ba681a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:10.370686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa3d92-56d0-4f78-8731-76b0d3ba681a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:10.377852 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:10.463874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 604/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:17:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:17:10.469763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca742c3-c6f3-4d4a-8d7c-7fee494a6161 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:10.564999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca742c3-c6f3-4d4a-8d7c-7fee494a6161 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:10.587824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:10.599098 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177528a-16ef-4f7a-9237-49e4c7f75a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:10.599751 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177528a-16ef-4f7a-9237-49e4c7f75a6c None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:10.600032 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177528a-16ef-4f7a-9237-49e4c7f75a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:10.600302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177528a-16ef-4f7a-9237-49e4c7f75a6c None admin] PATH_INFO: `/v3/projects/ab6d19dddb71417c869b40296e43ab84` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:10.601404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177528a-16ef-4f7a-9237-49e4c7f75a6c None admin] RBAC: Authorizing `identity:get_project(project_id=ab6d19dddb71417c869b40296e43ab84)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:10.602856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177528a-16ef-4f7a-9237-49e4c7f75a6c None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:10.604508 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 580/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:17:10 2022] GET /identity/v3/projects/ab6d19dddb71417c869b40296e43ab84 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:17:10.610711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa3d92-56d0-4f78-8731-76b0d3ba681a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:10.704204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa3d92-56d0-4f78-8731-76b0d3ba681a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:10.711526 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:10.736543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63590d3-0f8f-4ccd-bbc2-9ec3dd2deef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:10.737643 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63590d3-0f8f-4ccd-bbc2-9ec3dd2deef4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:10.737918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63590d3-0f8f-4ccd-bbc2-9ec3dd2deef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:10.738186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63590d3-0f8f-4ccd-bbc2-9ec3dd2deef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:10.738887 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63590d3-0f8f-4ccd-bbc2-9ec3dd2deef4 None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=ab6d19dddb71417c869b40296e43ab84, user.id=0031ad8c213a4d3ca0f7e9ed9ec0fb34) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:17:10.739190 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63590d3-0f8f-4ccd-bbc2-9ec3dd2deef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:10.740620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63590d3-0f8f-4ccd-bbc2-9ec3dd2deef4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:10.745698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 605/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:17:10 2022] GET /identity/v3/role_assignments?user.id=0031ad8c213a4d3ca0f7e9ed9ec0fb34&scope.project.id=ab6d19dddb71417c869b40296e43ab84&role.id=23538dc4c770454e80dd11ed8e498539 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:17:12.851864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a2f91-0e91-4a84-9fda-d501c6906063 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:12.852397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a2f91-0e91-4a84-9fda-d501c6906063 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:12.852397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a2f91-0e91-4a84-9fda-d501c6906063 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:12.853746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 581/1186] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:17:12.858908 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d0b84-3b97-40a7-8cd1-dc17f63a5f95 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:12.859306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d0b84-3b97-40a7-8cd1-dc17f63a5f95 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:12.859457 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d0b84-3b97-40a7-8cd1-dc17f63a5f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:12.876994 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3d7d0b84-3b97-40a7-8cd1-dc17f63a5f95 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:17:12.878651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3d7d0b84-3b97-40a7-8cd1-dc17f63a5f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:17:12.943331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 606/1187] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:17:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:17:13.209454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a2f91-0e91-4a84-9fda-d501c6906063 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:13.306711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a2f91-0e91-4a84-9fda-d501c6906063 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:13.339506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffb463-a6db-4741-af0c-20bc688ce41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:13.340145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffb463-a6db-4741-af0c-20bc688ce41a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:13.340426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffb463-a6db-4741-af0c-20bc688ce41a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:13.340699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffb463-a6db-4741-af0c-20bc688ce41a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:13.351435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffb463-a6db-4741-af0c-20bc688ce41a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-d4ffb463-a6db-4741-af0c-20bc688ce41a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:17:13.356263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:17:13.357422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:17:13.358473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:17:13.358473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:17:13.358473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:17:13.358473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:17:13.358473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:17:13.358473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:17:13.358473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:17:13.358473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:17:13.358473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:17:13.358473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:17:13.358473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:17:13.358473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:17:13.358473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 582/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:17:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:17:13.363942 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d0b84-3b97-40a7-8cd1-dc17f63a5f95 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:13.454435 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:13.457324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d0b84-3b97-40a7-8cd1-dc17f63a5f95 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:13.491357 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd9e4f-c5f8-4c10-a4a4-f97eb7e11c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:13.491921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd9e4f-c5f8-4c10-a4a4-f97eb7e11c5e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:13.492205 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd9e4f-c5f8-4c10-a4a4-f97eb7e11c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:13.492482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd9e4f-c5f8-4c10-a4a4-f97eb7e11c5e None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:13.493136 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd9e4f-c5f8-4c10-a4a4-f97eb7e11c5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:17:13.493440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd9e4f-c5f8-4c10-a4a4-f97eb7e11c5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:13.494796 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd9e4f-c5f8-4c10-a4a4-f97eb7e11c5e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:13.502637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 607/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:17:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:17:13.509019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffb463-a6db-4741-af0c-20bc688ce41a None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:13.560397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:13.605506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffb463-a6db-4741-af0c-20bc688ce41a None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:13.638543 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be816e-c022-4d85-a18a-63878d5c15f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:13.639163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be816e-c022-4d85-a18a-63878d5c15f1 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:13.639441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be816e-c022-4d85-a18a-63878d5c15f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:13.639744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be816e-c022-4d85-a18a-63878d5c15f1 None admin] PATH_INFO: `/v3/users/0031ad8c213a4d3ca0f7e9ed9ec0fb34` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:13.640897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be816e-c022-4d85-a18a-63878d5c15f1 None admin] RBAC: Authorizing `identity:get_user(user_id=0031ad8c213a4d3ca0f7e9ed9ec0fb34)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:13.642318 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be816e-c022-4d85-a18a-63878d5c15f1 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:13.644406 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 583/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:17:13 2022] GET /identity/v3/users/0031ad8c213a4d3ca0f7e9ed9ec0fb34 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:17:13.650404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd9e4f-c5f8-4c10-a4a4-f97eb7e11c5e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:13.745325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd9e4f-c5f8-4c10-a4a4-f97eb7e11c5e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:13.777052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee456835-91f4-445d-a2c4-b3893fe265d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:13.777662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee456835-91f4-445d-a2c4-b3893fe265d9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:13.777948 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee456835-91f4-445d-a2c4-b3893fe265d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:13.778214 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee456835-91f4-445d-a2c4-b3893fe265d9 None admin] PATH_INFO: `/v3/projects/ab6d19dddb71417c869b40296e43ab84` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:13.779309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee456835-91f4-445d-a2c4-b3893fe265d9 None admin] RBAC: Authorizing `identity:get_project(project_id=ab6d19dddb71417c869b40296e43ab84)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:13.780752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee456835-91f4-445d-a2c4-b3893fe265d9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:13.782586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 608/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:17:13 2022] GET /identity/v3/projects/ab6d19dddb71417c869b40296e43ab84 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:17:13.788326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be816e-c022-4d85-a18a-63878d5c15f1 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:13.884430 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be816e-c022-4d85-a18a-63878d5c15f1 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:13.903267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:13.917617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41d8bc-09ec-4459-b496-421f7b626a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:13.918354 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41d8bc-09ec-4459-b496-421f7b626a7e None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:13.918541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41d8bc-09ec-4459-b496-421f7b626a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:13.918815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41d8bc-09ec-4459-b496-421f7b626a7e None admin] PATH_INFO: `/v3/projects/ab6d19dddb71417c869b40296e43ab84/users/0031ad8c213a4d3ca0f7e9ed9ec0fb34/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:13.920700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41d8bc-09ec-4459-b496-421f7b626a7e None admin] RBAC: Authorizing `identity:create_grant(project_id=ab6d19dddb71417c869b40296e43ab84, user_id=0031ad8c213a4d3ca0f7e9ed9ec0fb34, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:13.922127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41d8bc-09ec-4459-b496-421f7b626a7e None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:13.934938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 584/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:17:13 2022] PUT /identity/v3/projects/ab6d19dddb71417c869b40296e43ab84/users/0031ad8c213a4d3ca0f7e9ed9ec0fb34/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:17:16.054743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d8f9b-f6c7-4f33-9e69-414529b23592 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:16.055255 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d8f9b-f6c7-4f33-9e69-414529b23592 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:16.055368 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d8f9b-f6c7-4f33-9e69-414529b23592 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:16.056653 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 609/1193] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:17:16.061920 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd296a6-f162-4442-ae5a-de7583b67a68 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:16.062339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd296a6-f162-4442-ae5a-de7583b67a68 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:16.062475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd296a6-f162-4442-ae5a-de7583b67a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:16.080674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-ddd296a6-f162-4442-ae5a-de7583b67a68 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:17:16.081849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ddd296a6-f162-4442-ae5a-de7583b67a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:17:16.145137 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 585/1194] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:17:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:17:16.415289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d8f9b-f6c7-4f33-9e69-414529b23592 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:16.445165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:16.513565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d8f9b-f6c7-4f33-9e69-414529b23592 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:16.545304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91ca6b-8812-4e5b-9f12-edb9ffa3d050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:16.545912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91ca6b-8812-4e5b-9f12-edb9ffa3d050 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:16.546189 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91ca6b-8812-4e5b-9f12-edb9ffa3d050 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:16.546459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91ca6b-8812-4e5b-9f12-edb9ffa3d050 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:16.556979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91ca6b-8812-4e5b-9f12-edb9ffa3d050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-9f91ca6b-8812-4e5b-9f12-edb9ffa3d050 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:17:16.561712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:17:16.562836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:17:16.563931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:17:16.563931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:17:16.563931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:17:16.563931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:17:16.563931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:17:16.563931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:17:16.563931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:17:16.563931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:17:16.563931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:17:16.563931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:17:16.563931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:17:16.563931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:17:16.563931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 610/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:17:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:17:16.568888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd296a6-f162-4442-ae5a-de7583b67a68 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:16.666195 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd296a6-f162-4442-ae5a-de7583b67a68 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:16.676396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:16.700132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbb776-52c0-4951-aabd-7dd083932076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:16.700727 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbb776-52c0-4951-aabd-7dd083932076 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:16.701006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbb776-52c0-4951-aabd-7dd083932076 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:16.701277 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbb776-52c0-4951-aabd-7dd083932076 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:16.701932 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbb776-52c0-4951-aabd-7dd083932076 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:17:16.702234 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbb776-52c0-4951-aabd-7dd083932076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:16.703619 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbb776-52c0-4951-aabd-7dd083932076 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:16.712018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 586/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:17:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:17:16.717968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91ca6b-8812-4e5b-9f12-edb9ffa3d050 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:16.804729 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:16.814621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91ca6b-8812-4e5b-9f12-edb9ffa3d050 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:16.847305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208b715-16bf-4c65-839f-85baffa2a78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:16.847981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b715-16bf-4c65-839f-85baffa2a78b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:16.848264 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b715-16bf-4c65-839f-85baffa2a78b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:16.848540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b715-16bf-4c65-839f-85baffa2a78b None admin] PATH_INFO: `/v3/users/0031ad8c213a4d3ca0f7e9ed9ec0fb34` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:16.849673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208b715-16bf-4c65-839f-85baffa2a78b None admin] RBAC: Authorizing `identity:get_user(user_id=0031ad8c213a4d3ca0f7e9ed9ec0fb34)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:16.851095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208b715-16bf-4c65-839f-85baffa2a78b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:16.852685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 611/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Sep 12 23:17:16 2022] GET /identity/v3/users/0031ad8c213a4d3ca0f7e9ed9ec0fb34 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:17:16.858277 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbb776-52c0-4951-aabd-7dd083932076 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:16.954017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbb776-52c0-4951-aabd-7dd083932076 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:16.986108 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589d0d1-c1a8-44bc-841b-3eae1ecd8cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:16.986724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589d0d1-c1a8-44bc-841b-3eae1ecd8cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:16.986997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589d0d1-c1a8-44bc-841b-3eae1ecd8cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:16.987266 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589d0d1-c1a8-44bc-841b-3eae1ecd8cb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:17.025384 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:17.051217 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589d0d1-c1a8-44bc-841b-3eae1ecd8cb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:17.052682 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589d0d1-c1a8-44bc-841b-3eae1ecd8cb8 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:17.145912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 587/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:17:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:17:17.151862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208b715-16bf-4c65-839f-85baffa2a78b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:17.246252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208b715-16bf-4c65-839f-85baffa2a78b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:17.278708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b7675-fe10-4a25-9722-db8afe60afb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:17.279324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b7675-fe10-4a25-9722-db8afe60afb7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:17.279629 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b7675-fe10-4a25-9722-db8afe60afb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:17.279905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b7675-fe10-4a25-9722-db8afe60afb7 None admin] PATH_INFO: `/v3/projects/ab6d19dddb71417c869b40296e43ab84` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:17.281291 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b7675-fe10-4a25-9722-db8afe60afb7 None admin] RBAC: Authorizing `identity:get_project(project_id=ab6d19dddb71417c869b40296e43ab84)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:17.282709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b7675-fe10-4a25-9722-db8afe60afb7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:17.284353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 612/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Sep 12 23:17:17 2022] GET /identity/v3/projects/ab6d19dddb71417c869b40296e43ab84 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:17:17.290117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589d0d1-c1a8-44bc-841b-3eae1ecd8cb8 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:17:17.356645 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:17.384896 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589d0d1-c1a8-44bc-841b-3eae1ecd8cb8 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:17:17.418448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf837c-b7d7-4213-9af0-895280576580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:17:17.419039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf837c-b7d7-4213-9af0-895280576580 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:17:17.419319 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf837c-b7d7-4213-9af0-895280576580 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:17:17.419610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf837c-b7d7-4213-9af0-895280576580 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:17:17.420322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf837c-b7d7-4213-9af0-895280576580 None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=ab6d19dddb71417c869b40296e43ab84, user.id=0031ad8c213a4d3ca0f7e9ed9ec0fb34) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:17:17.420627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf837c-b7d7-4213-9af0-895280576580 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:17:17.422049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf837c-b7d7-4213-9af0-895280576580 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:17:17.428067 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 588/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:17:17 2022] GET /identity/v3/role_assignments?user.id=0031ad8c213a4d3ca0f7e9ed9ec0fb34&scope.project.id=ab6d19dddb71417c869b40296e43ab84&role.id=23538dc4c770454e80dd11ed8e498539 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:17:27.274802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=87 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:17:27.425539 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=32 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:12.964529 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646ece-d49a-4197-85a6-af4c7bceb5a8 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:12.965124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646ece-d49a-4197-85a6-af4c7bceb5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:12.965124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646ece-d49a-4197-85a6-af4c7bceb5a8 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:12.966650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 613/1201] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:18:12.973452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cda18-63dc-490a-898c-8c8d7bd0d40e None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:12.974025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cda18-63dc-490a-898c-8c8d7bd0d40e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:12.974025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cda18-63dc-490a-898c-8c8d7bd0d40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:12.995268 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-6f5cda18-63dc-490a-898c-8c8d7bd0d40e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:18:12.996503 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6f5cda18-63dc-490a-898c-8c8d7bd0d40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:18:13.087644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 589/1202] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:18:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:18:13.388697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60646ece-d49a-4197-85a6-af4c7bceb5a8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:13.514994 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60646ece-d49a-4197-85a6-af4c7bceb5a8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:13.563547 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890b8ef-00ec-4048-b133-c5fa12d7c38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:13.564235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890b8ef-00ec-4048-b133-c5fa12d7c38c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:13.564491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890b8ef-00ec-4048-b133-c5fa12d7c38c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:13.564769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890b8ef-00ec-4048-b133-c5fa12d7c38c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:13.580039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890b8ef-00ec-4048-b133-c5fa12d7c38c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:13.582179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890b8ef-00ec-4048-b133-c5fa12d7c38c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f890b8ef-00ec-4048-b133-c5fa12d7c38c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:18:13.603925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:13.607964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:18:13.609647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:18:13.609647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:13.609647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:13.609647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 614/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:13.612653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cda18-63dc-490a-898c-8c8d7bd0d40e None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:13.735915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cda18-63dc-490a-898c-8c8d7bd0d40e None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:13.770303 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:13.777081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a954e5-a0d8-4e40-8b8a-2fb06271e8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:13.777770 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a954e5-a0d8-4e40-8b8a-2fb06271e8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:13.778147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a954e5-a0d8-4e40-8b8a-2fb06271e8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:13.778492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a954e5-a0d8-4e40-8b8a-2fb06271e8b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:13.779230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a954e5-a0d8-4e40-8b8a-2fb06271e8b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:13.779627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a954e5-a0d8-4e40-8b8a-2fb06271e8b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:13.781918 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a954e5-a0d8-4e40-8b8a-2fb06271e8b5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:13.797241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 590/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:13.803099 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890b8ef-00ec-4048-b133-c5fa12d7c38c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:13.866197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:13.942568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890b8ef-00ec-4048-b133-c5fa12d7c38c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:13.993564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c6827-0b9a-4d5d-adf9-fca88063bc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:13.994233 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c6827-0b9a-4d5d-adf9-fca88063bc65 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:13.994466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c6827-0b9a-4d5d-adf9-fca88063bc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:13.994737 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c6827-0b9a-4d5d-adf9-fca88063bc65 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:13.995402 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c6827-0b9a-4d5d-adf9-fca88063bc65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:13.996849 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c6827-0b9a-4d5d-adf9-fca88063bc65 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:14.030539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 615/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Sep 12 23:18:13 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:18:16.509282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c385-4e2a-4746-bf05-13730e08bb53 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:16.510325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c385-4e2a-4746-bf05-13730e08bb53 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:16.510325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c385-4e2a-4746-bf05-13730e08bb53 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:16.511435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 591/1206] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:18:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:18:16.519153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412525f-7c7c-41eb-9fa6-d81a4d63a037 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:16.519851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412525f-7c7c-41eb-9fa6-d81a4d63a037 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:16.519851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412525f-7c7c-41eb-9fa6-d81a4d63a037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:16.542094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a412525f-7c7c-41eb-9fa6-d81a4d63a037 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:18:16.543559 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a412525f-7c7c-41eb-9fa6-d81a4d63a037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:18:16.629997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 616/1207] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:18:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:18:16.900453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2c385-4e2a-4746-bf05-13730e08bb53 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:17.042434 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2c385-4e2a-4746-bf05-13730e08bb53 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:17.082408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d57afd-147f-4470-9fdd-ca35b3f44130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:17.083077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d57afd-147f-4470-9fdd-ca35b3f44130 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:17.083386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d57afd-147f-4470-9fdd-ca35b3f44130 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:17.083663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d57afd-147f-4470-9fdd-ca35b3f44130 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:17.095387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d57afd-147f-4470-9fdd-ca35b3f44130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-58d57afd-147f-4470-9fdd-ca35b3f44130 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:18:17.101292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:17.102473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:17.103881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:17.103881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:17.103881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:17.103881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:17.103881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:18:17.103881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:18:17.103881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:18:17.103881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:18:17.103881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:18:17.103881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:18:17.103881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:18:17.103881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:17.103881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 592/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:18:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:17.108774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412525f-7c7c-41eb-9fa6-d81a4d63a037 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:17.160000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:17.237831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412525f-7c7c-41eb-9fa6-d81a4d63a037 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:17.280433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9b3c2-ba3c-40d3-aee3-6c33bb18413e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:17.281231 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9b3c2-ba3c-40d3-aee3-6c33bb18413e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:17.281513 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9b3c2-ba3c-40d3-aee3-6c33bb18413e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:17.281814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9b3c2-ba3c-40d3-aee3-6c33bb18413e None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:17.282868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9b3c2-ba3c-40d3-aee3-6c33bb18413e None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:17.283175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9b3c2-ba3c-40d3-aee3-6c33bb18413e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:17.284980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9b3c2-ba3c-40d3-aee3-6c33bb18413e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:17.296041 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 617/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:18:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:17.302572 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d57afd-147f-4470-9fdd-ca35b3f44130 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:17.366100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:17.434386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d57afd-147f-4470-9fdd-ca35b3f44130 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:17.473576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221c404-4ee9-48d5-8a2d-8d80e7f2bcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:17.474349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221c404-4ee9-48d5-8a2d-8d80e7f2bcfa None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:17.474638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221c404-4ee9-48d5-8a2d-8d80e7f2bcfa None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:17.474968 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221c404-4ee9-48d5-8a2d-8d80e7f2bcfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:17.486823 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221c404-4ee9-48d5-8a2d-8d80e7f2bcfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:17.488307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221c404-4ee9-48d5-8a2d-8d80e7f2bcfa None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-b221c404-4ee9-48d5-8a2d-8d80e7f2bcfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:18:17.505368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:17.506510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:18:17.507401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:18:17.507401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:17.507401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:17.507401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 593/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:18:17.512788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9b3c2-ba3c-40d3-aee3-6c33bb18413e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:17.644998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:17.649473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9b3c2-ba3c-40d3-aee3-6c33bb18413e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:17.690123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13204f89-44aa-4241-8d72-04474d5892db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:17.690686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13204f89-44aa-4241-8d72-04474d5892db None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:17.690947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13204f89-44aa-4241-8d72-04474d5892db None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:17.691198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13204f89-44aa-4241-8d72-04474d5892db None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:17.691871 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13204f89-44aa-4241-8d72-04474d5892db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:17.692254 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13204f89-44aa-4241-8d72-04474d5892db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:17.693712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13204f89-44aa-4241-8d72-04474d5892db None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:17.708605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 618/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:17.716808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221c404-4ee9-48d5-8a2d-8d80e7f2bcfa None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:17.829511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221c404-4ee9-48d5-8a2d-8d80e7f2bcfa None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:17.831505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:17.870274 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f3d78-b842-4454-a97f-6a3464b287f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:17.870902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f3d78-b842-4454-a97f-6a3464b287f2 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:17.871179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f3d78-b842-4454-a97f-6a3464b287f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:17.871448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f3d78-b842-4454-a97f-6a3464b287f2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:17.879995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f3d78-b842-4454-a97f-6a3464b287f2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:17.881533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f3d78-b842-4454-a97f-6a3464b287f2 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-471f3d78-b842-4454-a97f-6a3464b287f2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:18:17.893203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:17.894331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:17.896008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:17.896008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:17.896008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:18:17.896008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:18:17.896008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:18:17.896008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:18:17.896008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:18:17.896008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:18:17.896008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:18:17.896008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:17.896008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 594/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:17.900069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13204f89-44aa-4241-8d72-04474d5892db None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:18.014337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13204f89-44aa-4241-8d72-04474d5892db None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:18.055178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d82bd-6783-4c40-94f6-0e86eeea5973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:18.055817 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d82bd-6783-4c40-94f6-0e86eeea5973 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:18.056102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d82bd-6783-4c40-94f6-0e86eeea5973 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:18.056379 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d82bd-6783-4c40-94f6-0e86eeea5973 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:18.057124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d82bd-6783-4c40-94f6-0e86eeea5973 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:18.057439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d82bd-6783-4c40-94f6-0e86eeea5973 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:18.058942 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d82bd-6783-4c40-94f6-0e86eeea5973 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:18.079522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 619/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Sep 12 23:18:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:18.086557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f3d78-b842-4454-a97f-6a3464b287f2 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:18.199703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f3d78-b842-4454-a97f-6a3464b287f2 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:18.240677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365ef6d-e70f-4f9f-873e-801a11de5750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:18.241360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365ef6d-e70f-4f9f-873e-801a11de5750 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:18.241648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365ef6d-e70f-4f9f-873e-801a11de5750 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:18.241924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365ef6d-e70f-4f9f-873e-801a11de5750 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:18.276690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:18.328638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365ef6d-e70f-4f9f-873e-801a11de5750 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:18.330376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365ef6d-e70f-4f9f-873e-801a11de5750 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:18.456365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 595/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:18:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 23:18:18.458399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d82bd-6783-4c40-94f6-0e86eeea5973 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:18.478215 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:18.567498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d82bd-6783-4c40-94f6-0e86eeea5973 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:18.619336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda014b-0c1c-4c9e-9244-e4195cc00d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:18.619937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda014b-0c1c-4c9e-9244-e4195cc00d6b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:18.620245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda014b-0c1c-4c9e-9244-e4195cc00d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:18.620518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda014b-0c1c-4c9e-9244-e4195cc00d6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:18.636468 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda014b-0c1c-4c9e-9244-e4195cc00d6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:18.638407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda014b-0c1c-4c9e-9244-e4195cc00d6b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ebda014b-0c1c-4c9e-9244-e4195cc00d6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:18:18.660743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:18.662473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:18:18.663388 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:18:18.663388 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:18.663388 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:18.663388 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 620/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:18.671173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365ef6d-e70f-4f9f-873e-801a11de5750 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:18.745679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:18.795475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365ef6d-e70f-4f9f-873e-801a11de5750 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:18.837975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dd23e-4fab-4f7d-8b13-24f98e8f7433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:18.838625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dd23e-4fab-4f7d-8b13-24f98e8f7433 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:18.838949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dd23e-4fab-4f7d-8b13-24f98e8f7433 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:18.839288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dd23e-4fab-4f7d-8b13-24f98e8f7433 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:18.840074 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dd23e-4fab-4f7d-8b13-24f98e8f7433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:18.840481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dd23e-4fab-4f7d-8b13-24f98e8f7433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:18.842057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dd23e-4fab-4f7d-8b13-24f98e8f7433 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:18.856239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 596/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:18:18.863229 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda014b-0c1c-4c9e-9244-e4195cc00d6b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:18.957955 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:18.998385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda014b-0c1c-4c9e-9244-e4195cc00d6b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:19.036286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc1828-cb36-43c1-882d-f95c8c3c740d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:19.037136 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc1828-cb36-43c1-882d-f95c8c3c740d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:19.037450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc1828-cb36-43c1-882d-f95c8c3c740d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:19.037774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc1828-cb36-43c1-882d-f95c8c3c740d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:19.045436 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc1828-cb36-43c1-882d-f95c8c3c740d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:19.047493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc1828-cb36-43c1-882d-f95c8c3c740d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-0cdc1828-cb36-43c1-882d-f95c8c3c740d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:18:19.060049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:19.061093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:19.067940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:18:19.067940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:18:19.067940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:18:19.067940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:18:19.067940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:18:19.067940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:18:19.067940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:18:19.067940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:19.067940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 621/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:18:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:18:19.068544 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dd23e-4fab-4f7d-8b13-24f98e8f7433 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:19.179478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dd23e-4fab-4f7d-8b13-24f98e8f7433 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:19.205703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:19.221549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc9cac-2475-4fed-8b24-c9055948c218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:19.222171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc9cac-2475-4fed-8b24-c9055948c218 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:19.222578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc9cac-2475-4fed-8b24-c9055948c218 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:19.222861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc9cac-2475-4fed-8b24-c9055948c218 None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:19.223704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc9cac-2475-4fed-8b24-c9055948c218 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:19.224002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc9cac-2475-4fed-8b24-c9055948c218 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:19.226337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc9cac-2475-4fed-8b24-c9055948c218 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:19.246560 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 597/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:18:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:19.248173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc1828-cb36-43c1-882d-f95c8c3c740d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:19.366320 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc1828-cb36-43c1-882d-f95c8c3c740d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:19.397984 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:19.405776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96f3d4-7030-488e-b3ee-03863f1a95b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:19.406347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96f3d4-7030-488e-b3ee-03863f1a95b7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:19.406635 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96f3d4-7030-488e-b3ee-03863f1a95b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:19.406898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96f3d4-7030-488e-b3ee-03863f1a95b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:19.407648 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96f3d4-7030-488e-b3ee-03863f1a95b7 None admin] RBAC: Adding query filter params (role.id=5c7e1f557ae04250a01db783f3c27a5f, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=2b3b0a0317b14e29a32eb6ba6b89c2aa) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:19.407956 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96f3d4-7030-488e-b3ee-03863f1a95b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:19.409374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96f3d4-7030-488e-b3ee-03863f1a95b7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:19.415236 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 622/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:18:19 2022] GET /identity/v3/role_assignments?user.id=2b3b0a0317b14e29a32eb6ba6b89c2aa&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=5c7e1f557ae04250a01db783f3c27a5f => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:21.610909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacde9d-9520-42de-9aab-246c02fd4a30 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:21.611558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacde9d-9520-42de-9aab-246c02fd4a30 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:21.611558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacde9d-9520-42de-9aab-246c02fd4a30 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:21.613034 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 598/1220] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:18:21.620246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910a658-d90d-4633-b3fc-0776acfdc59c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:21.620771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910a658-d90d-4633-b3fc-0776acfdc59c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:21.621002 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910a658-d90d-4633-b3fc-0776acfdc59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:21.645507 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f910a658-d90d-4633-b3fc-0776acfdc59c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:18:21.646752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f910a658-d90d-4633-b3fc-0776acfdc59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:18:21.720910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 623/1221] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:18:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:18:21.991472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bacde9d-9520-42de-9aab-246c02fd4a30 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:22.097781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bacde9d-9520-42de-9aab-246c02fd4a30 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:22.136483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95428f45-4483-4b47-978d-67733625a750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:22.136483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428f45-4483-4b47-978d-67733625a750 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:22.136483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428f45-4483-4b47-978d-67733625a750 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:22.136483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428f45-4483-4b47-978d-67733625a750 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:22.147884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95428f45-4483-4b47-978d-67733625a750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-95428f45-4483-4b47-978d-67733625a750 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:18:22.154238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:22.155704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:22.160773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:22.160773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:22.160773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:22.160773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:22.160773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:18:22.160773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:18:22.160773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:18:22.160773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:18:22.160773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:18:22.160773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:18:22.160773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:18:22.160773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:22.160773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 599/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:18:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:18:22.161967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910a658-d90d-4633-b3fc-0776acfdc59c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:22.272042 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910a658-d90d-4633-b3fc-0776acfdc59c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:22.298110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:22.312203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54697bbd-6787-489f-a6a0-4f0896499e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:22.312882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54697bbd-6787-489f-a6a0-4f0896499e1d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:22.313208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54697bbd-6787-489f-a6a0-4f0896499e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:22.313496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54697bbd-6787-489f-a6a0-4f0896499e1d None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:22.314272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54697bbd-6787-489f-a6a0-4f0896499e1d None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:22.314631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54697bbd-6787-489f-a6a0-4f0896499e1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:22.316211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54697bbd-6787-489f-a6a0-4f0896499e1d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:22.325264 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 624/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:18:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:22.331282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95428f45-4483-4b47-978d-67733625a750 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:22.379150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:22.441723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95428f45-4483-4b47-978d-67733625a750 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:22.480838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389a906-aff0-472a-8cf5-a81ac146f7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:22.481549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389a906-aff0-472a-8cf5-a81ac146f7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:22.481896 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389a906-aff0-472a-8cf5-a81ac146f7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:22.482218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389a906-aff0-472a-8cf5-a81ac146f7c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:22.494300 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389a906-aff0-472a-8cf5-a81ac146f7c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:22.495807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389a906-aff0-472a-8cf5-a81ac146f7c0 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-3389a906-aff0-472a-8cf5-a81ac146f7c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:18:22.510575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:22.511807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:18:22.512886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:18:22.512886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:22.512886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:22.512886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 600/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:22.517463 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54697bbd-6787-489f-a6a0-4f0896499e1d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:22.640537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54697bbd-6787-489f-a6a0-4f0896499e1d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:22.683065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69b57e-e26e-4df0-81fd-87e7962b4b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:22.683822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69b57e-e26e-4df0-81fd-87e7962b4b3f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:22.684131 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69b57e-e26e-4df0-81fd-87e7962b4b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:22.684529 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69b57e-e26e-4df0-81fd-87e7962b4b3f None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:22.685203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69b57e-e26e-4df0-81fd-87e7962b4b3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:22.685509 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69b57e-e26e-4df0-81fd-87e7962b4b3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:22.687158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69b57e-e26e-4df0-81fd-87e7962b4b3f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:22.701142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 625/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:22.707532 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389a906-aff0-472a-8cf5-a81ac146f7c0 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:22.807985 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:22.815690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389a906-aff0-472a-8cf5-a81ac146f7c0 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:22.857002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7483a1e-8342-4c62-84c2-b3b07f321d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:22.857712 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7483a1e-8342-4c62-84c2-b3b07f321d32 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:22.858058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7483a1e-8342-4c62-84c2-b3b07f321d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:22.858375 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7483a1e-8342-4c62-84c2-b3b07f321d32 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:22.866478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7483a1e-8342-4c62-84c2-b3b07f321d32 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:22.868059 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7483a1e-8342-4c62-84c2-b3b07f321d32 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-b7483a1e-8342-4c62-84c2-b3b07f321d32 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:18:22.879618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:22.885552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:22.886606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:22.886606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:22.886606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:18:22.886606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:18:22.886606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:18:22.886606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:18:22.886606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:18:22.886606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:18:22.886606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:18:22.886606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:22.886606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 601/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:22.887308 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69b57e-e26e-4df0-81fd-87e7962b4b3f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:22.939098 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:23.000498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69b57e-e26e-4df0-81fd-87e7962b4b3f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:23.038945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ad6f4-2360-4155-a3bb-0dc11074e03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:23.039816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ad6f4-2360-4155-a3bb-0dc11074e03a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:23.040142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ad6f4-2360-4155-a3bb-0dc11074e03a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:23.040460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ad6f4-2360-4155-a3bb-0dc11074e03a None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:23.041330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ad6f4-2360-4155-a3bb-0dc11074e03a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:23.041692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ad6f4-2360-4155-a3bb-0dc11074e03a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:23.043320 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ad6f4-2360-4155-a3bb-0dc11074e03a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:23.062243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 626/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Sep 12 23:18:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:23.068592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7483a1e-8342-4c62-84c2-b3b07f321d32 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:23.171400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7483a1e-8342-4c62-84c2-b3b07f321d32 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:23.208976 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a8d37-32db-4a2a-87a2-17180f4403b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:23.209608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a8d37-32db-4a2a-87a2-17180f4403b5 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:23.209888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a8d37-32db-4a2a-87a2-17180f4403b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:23.210158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a8d37-32db-4a2a-87a2-17180f4403b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:23.221024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a8d37-32db-4a2a-87a2-17180f4403b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:23.222502 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a8d37-32db-4a2a-87a2-17180f4403b5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-ab8a8d37-32db-4a2a-87a2-17180f4403b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:18:23.238506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:23.244510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:18:23.245467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:18:23.245467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:23.245467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:23.245467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 602/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:23.245712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ad6f4-2360-4155-a3bb-0dc11074e03a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:23.364654 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ad6f4-2360-4155-a3bb-0dc11074e03a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:23.367265 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:23.402916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3db4fe-ad42-46bd-b59e-88a7cb9e3bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:23.403538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3db4fe-ad42-46bd-b59e-88a7cb9e3bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:23.403851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3db4fe-ad42-46bd-b59e-88a7cb9e3bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:23.404130 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3db4fe-ad42-46bd-b59e-88a7cb9e3bb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:23.405050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3db4fe-ad42-46bd-b59e-88a7cb9e3bb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:23.405389 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3db4fe-ad42-46bd-b59e-88a7cb9e3bb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:23.406806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3db4fe-ad42-46bd-b59e-88a7cb9e3bb5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:23.421615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 627/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:18:23.427955 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a8d37-32db-4a2a-87a2-17180f4403b5 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:23.439828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:23.535986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a8d37-32db-4a2a-87a2-17180f4403b5 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:23.569385 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b135bb-e4ee-40b0-b3dd-a22c09766c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:23.570021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b135bb-e4ee-40b0-b3dd-a22c09766c6f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:23.570357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b135bb-e4ee-40b0-b3dd-a22c09766c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:23.570593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b135bb-e4ee-40b0-b3dd-a22c09766c6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:23.577993 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b135bb-e4ee-40b0-b3dd-a22c09766c6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:23.579448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b135bb-e4ee-40b0-b3dd-a22c09766c6f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-07b135bb-e4ee-40b0-b3dd-a22c09766c6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:18:23.592427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:23.594644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:23.597040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:18:23.597040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:18:23.597040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:18:23.597040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:18:23.597040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:18:23.597040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:18:23.597040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:18:23.597040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:23.597040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 603/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:18:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:23.600502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3db4fe-ad42-46bd-b59e-88a7cb9e3bb5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:23.717167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3db4fe-ad42-46bd-b59e-88a7cb9e3bb5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:23.754897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a32f7-07c0-4ab7-8fc8-676baf4797f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:23.755781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a32f7-07c0-4ab7-8fc8-676baf4797f5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:23.756068 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a32f7-07c0-4ab7-8fc8-676baf4797f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:23.756348 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a32f7-07c0-4ab7-8fc8-676baf4797f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:23.757038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a32f7-07c0-4ab7-8fc8-676baf4797f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:23.757349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a32f7-07c0-4ab7-8fc8-676baf4797f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:23.758775 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a32f7-07c0-4ab7-8fc8-676baf4797f5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:23.771976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 628/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:18:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:18:23.778217 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b135bb-e4ee-40b0-b3dd-a22c09766c6f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:23.855387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:23.895395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b135bb-e4ee-40b0-b3dd-a22c09766c6f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:23.932311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01779efb-6f96-45cf-81e5-b59f05e16f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:23.932972 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01779efb-6f96-45cf-81e5-b59f05e16f1b None admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:23.933248 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01779efb-6f96-45cf-81e5-b59f05e16f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:23.933522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01779efb-6f96-45cf-81e5-b59f05e16f1b None admin] PATH_INFO: `/v3/projects/32bbb936501846e8914acf07890c2891/users/2b3b0a0317b14e29a32eb6ba6b89c2aa/roles/5c7e1f557ae04250a01db783f3c27a5f` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:23.949826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01779efb-6f96-45cf-81e5-b59f05e16f1b None admin] RBAC: Authorizing `identity:create_grant(project_id=32bbb936501846e8914acf07890c2891, user_id=2b3b0a0317b14e29a32eb6ba6b89c2aa, role_id=5c7e1f557ae04250a01db783f3c27a5f)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:23.951282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01779efb-6f96-45cf-81e5-b59f05e16f1b None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:23.964162 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 604/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:18:23 2022] PUT /identity/v3/projects/32bbb936501846e8914acf07890c2891/users/2b3b0a0317b14e29a32eb6ba6b89c2aa/roles/5c7e1f557ae04250a01db783f3c27a5f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:18:26.179243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf884d-d526-4c31-832d-d351bf4409f0 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:26.179795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf884d-d526-4c31-832d-d351bf4409f0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:26.179795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf884d-d526-4c31-832d-d351bf4409f0 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:26.181139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 629/1233] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:18:26.187228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e42b40-97e3-4db3-9634-9b92634262b2 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:26.187659 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e42b40-97e3-4db3-9634-9b92634262b2 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:26.187819 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e42b40-97e3-4db3-9634-9b92634262b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:26.208717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-15e42b40-97e3-4db3-9634-9b92634262b2 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:18:26.209919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-15e42b40-97e3-4db3-9634-9b92634262b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:18:26.288236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 605/1234] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:18:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:18:26.564984 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf884d-d526-4c31-832d-d351bf4409f0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:26.594204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:26.692221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf884d-d526-4c31-832d-d351bf4409f0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:26.732643 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe658d05-5798-47e2-bec9-5ed71c173b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:26.733365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe658d05-5798-47e2-bec9-5ed71c173b14 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:26.733694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe658d05-5798-47e2-bec9-5ed71c173b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:26.733973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe658d05-5798-47e2-bec9-5ed71c173b14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:26.744939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe658d05-5798-47e2-bec9-5ed71c173b14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-fe658d05-5798-47e2-bec9-5ed71c173b14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:18:26.751187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:26.752455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:26.754089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:26.754089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:26.754089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:26.754089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:26.754089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:18:26.754089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:18:26.754089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:18:26.754089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:18:26.754089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:18:26.754089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:18:26.754089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:18:26.754089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:26.754089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 630/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Sep 12 23:18:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:18:26.759744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e42b40-97e3-4db3-9634-9b92634262b2 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:26.809379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:26.873327 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e42b40-97e3-4db3-9634-9b92634262b2 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:26.910093 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d220cd-7a8c-433b-a0c3-40c6cdd01b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:26.910700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d220cd-7a8c-433b-a0c3-40c6cdd01b11 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:26.910979 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d220cd-7a8c-433b-a0c3-40c6cdd01b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:26.911251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d220cd-7a8c-433b-a0c3-40c6cdd01b11 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:26.911939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d220cd-7a8c-433b-a0c3-40c6cdd01b11 None admin] RBAC: Adding query filter params (name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:26.912242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d220cd-7a8c-433b-a0c3-40c6cdd01b11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:26.916085 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d220cd-7a8c-433b-a0c3-40c6cdd01b11 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:26.928184 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 606/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Sep 12 23:18:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:26.930109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe658d05-5798-47e2-bec9-5ed71c173b14 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:27.026922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:27.043255 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe658d05-5798-47e2-bec9-5ed71c173b14 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:27.080461 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963de1dd-ec7f-474d-9584-c8ec433de7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:27.081086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963de1dd-ec7f-474d-9584-c8ec433de7da None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:27.081360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963de1dd-ec7f-474d-9584-c8ec433de7da None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:27.081625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963de1dd-ec7f-474d-9584-c8ec433de7da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:27.091851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963de1dd-ec7f-474d-9584-c8ec433de7da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:27.093290 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963de1dd-ec7f-474d-9584-c8ec433de7da None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-963de1dd-ec7f-474d-9584-c8ec433de7da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:18:27.107957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:27.109686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:18:27.110695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:18:27.110695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:27.110695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:27.110695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 631/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:27.115562 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d220cd-7a8c-433b-a0c3-40c6cdd01b11 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:27.223300 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d220cd-7a8c-433b-a0c3-40c6cdd01b11 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:27.231038 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:27.265274 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95e141-922f-4449-bd29-ee1db15d6ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:27.265913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95e141-922f-4449-bd29-ee1db15d6ecd None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:27.266200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95e141-922f-4449-bd29-ee1db15d6ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:27.266476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95e141-922f-4449-bd29-ee1db15d6ecd None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:27.267176 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95e141-922f-4449-bd29-ee1db15d6ecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:27.267509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95e141-922f-4449-bd29-ee1db15d6ecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:27.269026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95e141-922f-4449-bd29-ee1db15d6ecd None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:27.283165 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 607/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:27.289321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963de1dd-ec7f-474d-9584-c8ec433de7da None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:27.408875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963de1dd-ec7f-474d-9584-c8ec433de7da None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:27.447266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bd776-2f48-4d73-9fc2-6105988bd0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:27.448232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bd776-2f48-4d73-9fc2-6105988bd0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:27.448609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bd776-2f48-4d73-9fc2-6105988bd0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:27.448916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bd776-2f48-4d73-9fc2-6105988bd0c5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:27.452339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:27.459478 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bd776-2f48-4d73-9fc2-6105988bd0c5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:27.462186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bd776-2f48-4d73-9fc2-6105988bd0c5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6a7bd776-2f48-4d73-9fc2-6105988bd0c5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:18:27.475747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:27.476724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:27.477726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:27.477726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:27.477726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:18:27.477726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:18:27.477726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:18:27.477726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:18:27.477726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:18:27.477726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:18:27.477726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:18:27.477726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:27.477726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 632/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:27.483855 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95e141-922f-4449-bd29-ee1db15d6ecd None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:27.590111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95e141-922f-4449-bd29-ee1db15d6ecd None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:27.628711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836b003-aa4b-44d7-9a9f-2aa251688cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:27.629324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836b003-aa4b-44d7-9a9f-2aa251688cee None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:27.629604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836b003-aa4b-44d7-9a9f-2aa251688cee None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:27.629879 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836b003-aa4b-44d7-9a9f-2aa251688cee None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:27.630686 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836b003-aa4b-44d7-9a9f-2aa251688cee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:27.630978 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836b003-aa4b-44d7-9a9f-2aa251688cee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:27.632373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836b003-aa4b-44d7-9a9f-2aa251688cee None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:27.651614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 608/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Sep 12 23:18:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:27.658112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bd776-2f48-4d73-9fc2-6105988bd0c5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:27.767840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bd776-2f48-4d73-9fc2-6105988bd0c5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:27.805694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace96b7-fc7e-48f3-ad99-6ea221e4f0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:27.806309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace96b7-fc7e-48f3-ad99-6ea221e4f0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:27.806598 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace96b7-fc7e-48f3-ad99-6ea221e4f0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:27.806873 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace96b7-fc7e-48f3-ad99-6ea221e4f0e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:27.876730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:27.885054 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace96b7-fc7e-48f3-ad99-6ea221e4f0e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:27.886537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace96b7-fc7e-48f3-ad99-6ea221e4f0e0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:27.994480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 633/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:18:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:18:28.001045 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836b003-aa4b-44d7-9a9f-2aa251688cee None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:28.032179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:28.118657 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836b003-aa4b-44d7-9a9f-2aa251688cee None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:28.157988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51848d-0b64-4606-aede-37c9c31ba16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:28.158626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51848d-0b64-4606-aede-37c9c31ba16f None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:28.158909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51848d-0b64-4606-aede-37c9c31ba16f None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:28.159188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51848d-0b64-4606-aede-37c9c31ba16f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:28.169710 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51848d-0b64-4606-aede-37c9c31ba16f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:28.171817 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51848d-0b64-4606-aede-37c9c31ba16f None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-6c51848d-0b64-4606-aede-37c9c31ba16f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:18:28.187945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:28.189389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:18:28.190565 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:18:28.190565 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:28.190565 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:28.190565 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 609/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:28.194881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace96b7-fc7e-48f3-ad99-6ea221e4f0e0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:28.298139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:28.306214 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace96b7-fc7e-48f3-ad99-6ea221e4f0e0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:28.340426 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aef26-adc6-47ab-83fa-b088053dd82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:28.341115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aef26-adc6-47ab-83fa-b088053dd82b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:28.341399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aef26-adc6-47ab-83fa-b088053dd82b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:28.341712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aef26-adc6-47ab-83fa-b088053dd82b None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:28.342378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aef26-adc6-47ab-83fa-b088053dd82b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:28.343263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aef26-adc6-47ab-83fa-b088053dd82b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:28.344705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aef26-adc6-47ab-83fa-b088053dd82b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:28.356609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 634/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:28.362571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51848d-0b64-4606-aede-37c9c31ba16f None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:28.444681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:28.466580 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51848d-0b64-4606-aede-37c9c31ba16f None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:28.501379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0fa062-08a7-409b-84fe-8421dd39f6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:28.502011 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fa062-08a7-409b-84fe-8421dd39f6bb None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:28.502292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fa062-08a7-409b-84fe-8421dd39f6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:28.502572 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fa062-08a7-409b-84fe-8421dd39f6bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:28.509275 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0fa062-08a7-409b-84fe-8421dd39f6bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:28.510752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0fa062-08a7-409b-84fe-8421dd39f6bb None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-ed0fa062-08a7-409b-84fe-8421dd39f6bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:18:28.522891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:28.524051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:28.530540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:18:28.530540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:18:28.530540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:18:28.530540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:18:28.530540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:18:28.530540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:18:28.530540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:18:28.530540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:28.530540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 610/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:18:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:28.531152 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aef26-adc6-47ab-83fa-b088053dd82b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:28.645520 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aef26-adc6-47ab-83fa-b088053dd82b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:28.687035 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbee311-0e4e-4003-b805-7477cb62348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:28.687961 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbee311-0e4e-4003-b805-7477cb62348d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:28.688256 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbee311-0e4e-4003-b805-7477cb62348d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:28.688692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbee311-0e4e-4003-b805-7477cb62348d None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:28.689657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbee311-0e4e-4003-b805-7477cb62348d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:28.690122 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbee311-0e4e-4003-b805-7477cb62348d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:28.692480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbee311-0e4e-4003-b805-7477cb62348d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:28.706247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 635/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:18:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:28.712780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0fa062-08a7-409b-84fe-8421dd39f6bb None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:28.830588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0fa062-08a7-409b-84fe-8421dd39f6bb None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:28.870797 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e892599-8f07-4ec4-bf1b-333f4fdd5403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:28.871407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e892599-8f07-4ec4-bf1b-333f4fdd5403 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:28.871723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e892599-8f07-4ec4-bf1b-333f4fdd5403 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:28.871998 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e892599-8f07-4ec4-bf1b-333f4fdd5403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:28.872742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e892599-8f07-4ec4-bf1b-333f4fdd5403 None admin] RBAC: Adding query filter params (role.id=5c7e1f557ae04250a01db783f3c27a5f, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=2b3b0a0317b14e29a32eb6ba6b89c2aa) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:28.873035 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e892599-8f07-4ec4-bf1b-333f4fdd5403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:28.875020 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e892599-8f07-4ec4-bf1b-333f4fdd5403 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:28.880936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 611/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:18:28 2022] GET /identity/v3/role_assignments?user.id=2b3b0a0317b14e29a32eb6ba6b89c2aa&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=5c7e1f557ae04250a01db783f3c27a5f => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:31.306035 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4596b-739a-44f8-b30c-129222919a18 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:31.306674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4596b-739a-44f8-b30c-129222919a18 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:31.307002 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4596b-739a-44f8-b30c-129222919a18 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:31.310099 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 636/1247] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:18:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:18:31.315188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1982478-257a-4b4f-9941-22f119427c8e None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:31.315889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1982478-257a-4b4f-9941-22f119427c8e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:31.315889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1982478-257a-4b4f-9941-22f119427c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:31.324991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:31.340005 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b1982478-257a-4b4f-9941-22f119427c8e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:18:31.341212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b1982478-257a-4b4f-9941-22f119427c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:18:31.414902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 612/1248] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:18:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:18:31.696992 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4596b-739a-44f8-b30c-129222919a18 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:31.716234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:31.810529 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4596b-739a-44f8-b30c-129222919a18 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:31.852258 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f82d28-e9d9-4888-ac95-0492704993dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:31.853003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f82d28-e9d9-4888-ac95-0492704993dd None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:31.853309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f82d28-e9d9-4888-ac95-0492704993dd None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:31.853616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f82d28-e9d9-4888-ac95-0492704993dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:31.866383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f82d28-e9d9-4888-ac95-0492704993dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-44f82d28-e9d9-4888-ac95-0492704993dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:18:31.872593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:31.873754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:31.874826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:31.874826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:31.874826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:31.874826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:31.874826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:18:31.874826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:18:31.874826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:18:31.874826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:18:31.874826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:18:31.874826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:18:31.874826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:18:31.874826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:31.874826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 637/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:18:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:31.880754 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1982478-257a-4b4f-9941-22f119427c8e None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:31.992146 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1982478-257a-4b4f-9941-22f119427c8e None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:32.023016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:32.029435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133e411-e9f4-4f05-9dc2-564386e38004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:32.030682 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133e411-e9f4-4f05-9dc2-564386e38004 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:32.031022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133e411-e9f4-4f05-9dc2-564386e38004 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:32.031318 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133e411-e9f4-4f05-9dc2-564386e38004 None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:32.032152 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133e411-e9f4-4f05-9dc2-564386e38004 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:32.032613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133e411-e9f4-4f05-9dc2-564386e38004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:32.034032 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133e411-e9f4-4f05-9dc2-564386e38004 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:32.043123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 613/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:18:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:32.050029 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f82d28-e9d9-4888-ac95-0492704993dd None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:32.145933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:32.174365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f82d28-e9d9-4888-ac95-0492704993dd None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:32.216776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce31b2-0fa9-43c2-97d3-30f96ca1897d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:32.217979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce31b2-0fa9-43c2-97d3-30f96ca1897d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:32.218283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce31b2-0fa9-43c2-97d3-30f96ca1897d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:32.218568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce31b2-0fa9-43c2-97d3-30f96ca1897d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:32.229735 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce31b2-0fa9-43c2-97d3-30f96ca1897d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:32.231330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce31b2-0fa9-43c2-97d3-30f96ca1897d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-18ce31b2-0fa9-43c2-97d3-30f96ca1897d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:18:32.249188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:32.250335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:18:32.251703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:18:32.251703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:32.251703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:32.251703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 638/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:32.256256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133e411-e9f4-4f05-9dc2-564386e38004 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:32.362126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133e411-e9f4-4f05-9dc2-564386e38004 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:32.396177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17201daa-0388-4746-8ae5-459d80ab9e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:32.396766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17201daa-0388-4746-8ae5-459d80ab9e96 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:32.397047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17201daa-0388-4746-8ae5-459d80ab9e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:32.397318 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17201daa-0388-4746-8ae5-459d80ab9e96 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:32.397951 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17201daa-0388-4746-8ae5-459d80ab9e96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:32.398253 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17201daa-0388-4746-8ae5-459d80ab9e96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:32.399644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17201daa-0388-4746-8ae5-459d80ab9e96 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:32.412092 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 614/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:18:32.417802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce31b2-0fa9-43c2-97d3-30f96ca1897d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:32.524310 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce31b2-0fa9-43c2-97d3-30f96ca1897d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:32.554664 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:32.563539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00a8fb-2132-4020-a4f6-89dc3754e644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:32.564222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00a8fb-2132-4020-a4f6-89dc3754e644 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:32.564508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00a8fb-2132-4020-a4f6-89dc3754e644 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:32.564784 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00a8fb-2132-4020-a4f6-89dc3754e644 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:32.573349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00a8fb-2132-4020-a4f6-89dc3754e644 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:32.574809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00a8fb-2132-4020-a4f6-89dc3754e644 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5f00a8fb-2132-4020-a4f6-89dc3754e644 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:18:32.587844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:32.589163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:32.590369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:32.590369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:32.590369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:18:32.590369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:18:32.590369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:18:32.590369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:18:32.590369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:18:32.590369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:18:32.590369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:18:32.590369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:32.590369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 639/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:18:32.595800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17201daa-0388-4746-8ae5-459d80ab9e96 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:32.668622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:32.715488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17201daa-0388-4746-8ae5-459d80ab9e96 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:32.751853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90a590-4cc1-45d3-b6c8-0eee14aabbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:32.752483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90a590-4cc1-45d3-b6c8-0eee14aabbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:32.752762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90a590-4cc1-45d3-b6c8-0eee14aabbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:32.753036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90a590-4cc1-45d3-b6c8-0eee14aabbf9 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:32.753780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90a590-4cc1-45d3-b6c8-0eee14aabbf9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:32.754087 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90a590-4cc1-45d3-b6c8-0eee14aabbf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:32.755556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90a590-4cc1-45d3-b6c8-0eee14aabbf9 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:32.775206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 615/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Sep 12 23:18:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:32.781336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00a8fb-2132-4020-a4f6-89dc3754e644 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:32.884697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00a8fb-2132-4020-a4f6-89dc3754e644 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:32.922652 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08a5e0-20f3-4528-8209-55a941e141c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:32.923272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a5e0-20f3-4528-8209-55a941e141c0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:32.923554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a5e0-20f3-4528-8209-55a941e141c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:32.923851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a5e0-20f3-4528-8209-55a941e141c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:32.987210 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:33.003132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08a5e0-20f3-4528-8209-55a941e141c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:33.006916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08a5e0-20f3-4528-8209-55a941e141c0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:33.113699 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 640/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:18:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:18:33.120027 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90a590-4cc1-45d3-b6c8-0eee14aabbf9 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:33.219884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90a590-4cc1-45d3-b6c8-0eee14aabbf9 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:33.232393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:33.252617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720feeb8-119b-4362-8276-951ff80c6570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:33.253238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720feeb8-119b-4362-8276-951ff80c6570 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:33.253516 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720feeb8-119b-4362-8276-951ff80c6570 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:33.253790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720feeb8-119b-4362-8276-951ff80c6570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:33.263409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720feeb8-119b-4362-8276-951ff80c6570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:33.265412 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720feeb8-119b-4362-8276-951ff80c6570 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-720feeb8-119b-4362-8276-951ff80c6570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:18:33.281046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:33.282167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:18:33.283269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:18:33.283269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:33.283269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:33.283269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 616/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:33.288004 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08a5e0-20f3-4528-8209-55a941e141c0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:33.373708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:33.385597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08a5e0-20f3-4528-8209-55a941e141c0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:33.419232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abc1d6-2cef-4199-b602-c32a93abcf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:33.419859 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abc1d6-2cef-4199-b602-c32a93abcf00 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:33.420140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abc1d6-2cef-4199-b602-c32a93abcf00 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:33.420412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abc1d6-2cef-4199-b602-c32a93abcf00 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:33.421040 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86abc1d6-2cef-4199-b602-c32a93abcf00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:33.421341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86abc1d6-2cef-4199-b602-c32a93abcf00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:33.422682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86abc1d6-2cef-4199-b602-c32a93abcf00 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:33.433450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 641/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:33.439138 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720feeb8-119b-4362-8276-951ff80c6570 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:33.535758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720feeb8-119b-4362-8276-951ff80c6570 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:33.568019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01873d22-45f9-4782-a7e0-a053480f0d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:33.568632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01873d22-45f9-4782-a7e0-a053480f0d19 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:33.568913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01873d22-45f9-4782-a7e0-a053480f0d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:33.569185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01873d22-45f9-4782-a7e0-a053480f0d19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:33.575626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01873d22-45f9-4782-a7e0-a053480f0d19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:33.577724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01873d22-45f9-4782-a7e0-a053480f0d19 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-01873d22-45f9-4782-a7e0-a053480f0d19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:18:33.590745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:33.591853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:33.592996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:18:33.592996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:18:33.592996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:18:33.592996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:18:33.592996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:18:33.592996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:18:33.592996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:18:33.592996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:33.592996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 617/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:18:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:33.602424 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abc1d6-2cef-4199-b602-c32a93abcf00 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:33.702751 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abc1d6-2cef-4199-b602-c32a93abcf00 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:33.733998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259e322-0f25-46cd-8548-826e3f59df43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:33.734593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e322-0f25-46cd-8548-826e3f59df43 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:33.734872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e322-0f25-46cd-8548-826e3f59df43 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:33.735146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e322-0f25-46cd-8548-826e3f59df43 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:33.735850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259e322-0f25-46cd-8548-826e3f59df43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:33.736156 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259e322-0f25-46cd-8548-826e3f59df43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:33.737570 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259e322-0f25-46cd-8548-826e3f59df43 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:33.749627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 642/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:18:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:18:33.755701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01873d22-45f9-4782-a7e0-a053480f0d19 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:33.781257 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:33.852788 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01873d22-45f9-4782-a7e0-a053480f0d19 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:33.885068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a9dab-934a-4516-9e01-cac635bc8f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:33.885658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a9dab-934a-4516-9e01-cac635bc8f10 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:33.885938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a9dab-934a-4516-9e01-cac635bc8f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:33.886209 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a9dab-934a-4516-9e01-cac635bc8f10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:33.886924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a9dab-934a-4516-9e01-cac635bc8f10 None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=2b3b0a0317b14e29a32eb6ba6b89c2aa) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:33.887228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a9dab-934a-4516-9e01-cac635bc8f10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:33.888678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a9dab-934a-4516-9e01-cac635bc8f10 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:33.894385 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 618/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:18:33 2022] GET /identity/v3/role_assignments?user.id=2b3b0a0317b14e29a32eb6ba6b89c2aa&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=23538dc4c770454e80dd11ed8e498539 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:36.049024 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1926b70-887a-4d01-b542-0fd4fbd9df0a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:36.049597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1926b70-887a-4d01-b542-0fd4fbd9df0a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:36.049597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1926b70-887a-4d01-b542-0fd4fbd9df0a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:36.050873 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 643/1261] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:18:36.056016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b07f2-40f8-4f79-b96f-96dd12b7cb45 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:36.056381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b07f2-40f8-4f79-b96f-96dd12b7cb45 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:36.056571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b07f2-40f8-4f79-b96f-96dd12b7cb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:36.074729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-1a6b07f2-40f8-4f79-b96f-96dd12b7cb45 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:18:36.075949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1a6b07f2-40f8-4f79-b96f-96dd12b7cb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:18:36.144182 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 619/1262] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:18:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:18:36.409200 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1926b70-887a-4d01-b542-0fd4fbd9df0a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:36.419821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:36.507578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1926b70-887a-4d01-b542-0fd4fbd9df0a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:36.539918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4372348-afd4-48e2-99bf-2f79690e3e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:36.540533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4372348-afd4-48e2-99bf-2f79690e3e4b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:36.540814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4372348-afd4-48e2-99bf-2f79690e3e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:36.541088 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4372348-afd4-48e2-99bf-2f79690e3e4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:36.551233 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4372348-afd4-48e2-99bf-2f79690e3e4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c4372348-afd4-48e2-99bf-2f79690e3e4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:18:36.556077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:36.557175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:36.558223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:36.558223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:36.558223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:36.558223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:36.558223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:18:36.558223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:18:36.558223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:18:36.558223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:18:36.558223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:18:36.558223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:18:36.558223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:18:36.558223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:36.558223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 644/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:18:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:36.563166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b07f2-40f8-4f79-b96f-96dd12b7cb45 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:36.580799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:36.677157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b07f2-40f8-4f79-b96f-96dd12b7cb45 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:36.713843 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e046b-fa6d-4b01-952a-42dd3a2e643a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:36.714460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e046b-fa6d-4b01-952a-42dd3a2e643a None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:36.714740 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e046b-fa6d-4b01-952a-42dd3a2e643a None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:36.715013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e046b-fa6d-4b01-952a-42dd3a2e643a None admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:36.715698 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e046b-fa6d-4b01-952a-42dd3a2e643a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:36.716003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e046b-fa6d-4b01-952a-42dd3a2e643a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:36.717880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e046b-fa6d-4b01-952a-42dd3a2e643a None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:36.725587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 620/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:18:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:36.731322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4372348-afd4-48e2-99bf-2f79690e3e4b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:36.796312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:36.829120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4372348-afd4-48e2-99bf-2f79690e3e4b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:36.861860 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843abfa-a932-4acf-b64f-722401ec6c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:36.862459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843abfa-a932-4acf-b64f-722401ec6c4f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:36.862738 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843abfa-a932-4acf-b64f-722401ec6c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:36.863008 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843abfa-a932-4acf-b64f-722401ec6c4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:36.872964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f843abfa-a932-4acf-b64f-722401ec6c4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:36.874377 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f843abfa-a932-4acf-b64f-722401ec6c4f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f843abfa-a932-4acf-b64f-722401ec6c4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:18:36.887858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:36.889444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:18:36.890347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:18:36.890347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:36.890347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:36.890347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 645/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:36.895144 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e046b-fa6d-4b01-952a-42dd3a2e643a None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:36.975759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:36.995498 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e046b-fa6d-4b01-952a-42dd3a2e643a None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:37.031732 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33192c10-a561-41b9-bfdf-8fe40ffd3671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:37.032409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33192c10-a561-41b9-bfdf-8fe40ffd3671 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:37.032692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33192c10-a561-41b9-bfdf-8fe40ffd3671 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:37.032966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33192c10-a561-41b9-bfdf-8fe40ffd3671 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:37.033799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33192c10-a561-41b9-bfdf-8fe40ffd3671 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:37.034108 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33192c10-a561-41b9-bfdf-8fe40ffd3671 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:37.035605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33192c10-a561-41b9-bfdf-8fe40ffd3671 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:37.048341 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 621/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:37.054005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843abfa-a932-4acf-b64f-722401ec6c4f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:37.148968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843abfa-a932-4acf-b64f-722401ec6c4f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:37.168467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:37.181930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8dc45-9cb8-47f4-968f-53bedded8474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:37.182545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8dc45-9cb8-47f4-968f-53bedded8474 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:37.182823 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8dc45-9cb8-47f4-968f-53bedded8474 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:37.183094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8dc45-9cb8-47f4-968f-53bedded8474 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:37.190202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8dc45-9cb8-47f4-968f-53bedded8474 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:37.191641 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8dc45-9cb8-47f4-968f-53bedded8474 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c3d8dc45-9cb8-47f4-968f-53bedded8474 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:18:37.202861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:37.203943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:37.204903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:37.204903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:37.204903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:18:37.204903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:18:37.204903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:18:37.204903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:18:37.204903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:18:37.204903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:18:37.204903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:18:37.204903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:37.204903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 646/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:37.209610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33192c10-a561-41b9-bfdf-8fe40ffd3671 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:37.304276 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33192c10-a561-41b9-bfdf-8fe40ffd3671 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:37.337677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482e20d-377f-4991-97c6-7f1b6e404d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:37.338271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482e20d-377f-4991-97c6-7f1b6e404d49 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:37.338549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482e20d-377f-4991-97c6-7f1b6e404d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:37.338816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482e20d-377f-4991-97c6-7f1b6e404d49 None admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:37.339526 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482e20d-377f-4991-97c6-7f1b6e404d49 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:37.339856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482e20d-377f-4991-97c6-7f1b6e404d49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:37.341273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482e20d-377f-4991-97c6-7f1b6e404d49 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:37.354457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:37.363651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 622/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Sep 12 23:18:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:37.369399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8dc45-9cb8-47f4-968f-53bedded8474 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:37.465647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8dc45-9cb8-47f4-968f-53bedded8474 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:37.496872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8c665-8041-4f26-9698-d50edf0a4406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:37.497472 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8c665-8041-4f26-9698-d50edf0a4406 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:37.497752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8c665-8041-4f26-9698-d50edf0a4406 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:37.498026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8c665-8041-4f26-9698-d50edf0a4406 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:37.508603 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8c665-8041-4f26-9698-d50edf0a4406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:37.510019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8c665-8041-4f26-9698-d50edf0a4406 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-a5e8c665-8041-4f26-9698-d50edf0a4406 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:18:37.523660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:37.524591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:18:37.525572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:18:37.525572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:37.525572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:37.525572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 647/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:37.530861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482e20d-377f-4991-97c6-7f1b6e404d49 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:37.629310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482e20d-377f-4991-97c6-7f1b6e404d49 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:37.662844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4a5ca-f4ff-481e-8315-eed3282b3be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:37.663450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4a5ca-f4ff-481e-8315-eed3282b3be8 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:37.663758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4a5ca-f4ff-481e-8315-eed3282b3be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:37.664034 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4a5ca-f4ff-481e-8315-eed3282b3be8 None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:37.664666 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4a5ca-f4ff-481e-8315-eed3282b3be8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:37.664969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4a5ca-f4ff-481e-8315-eed3282b3be8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:37.666363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4a5ca-f4ff-481e-8315-eed3282b3be8 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:37.677115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 623/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:37.682900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8c665-8041-4f26-9698-d50edf0a4406 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:37.714667 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:37.781694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8c665-8041-4f26-9698-d50edf0a4406 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:37.813424 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd3bd1-8216-4c2a-b0b4-81533f0311b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:37.814035 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd3bd1-8216-4c2a-b0b4-81533f0311b3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:37.814318 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd3bd1-8216-4c2a-b0b4-81533f0311b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:37.814595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd3bd1-8216-4c2a-b0b4-81533f0311b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:37.820986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd3bd1-8216-4c2a-b0b4-81533f0311b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:37.822410 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd3bd1-8216-4c2a-b0b4-81533f0311b3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-17cd3bd1-8216-4c2a-b0b4-81533f0311b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:18:37.834225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:37.835253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:37.836395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:18:37.836395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:18:37.836395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:18:37.836395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:18:37.836395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:18:37.836395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:18:37.836395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:18:37.836395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:37.836395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 648/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:18:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:37.841118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4a5ca-f4ff-481e-8315-eed3282b3be8 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:37.897761 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:37.939310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4a5ca-f4ff-481e-8315-eed3282b3be8 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:37.971693 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e251c2-a02b-41a8-bdaa-96dcb7016e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:37.972286 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e251c2-a02b-41a8-bdaa-96dcb7016e76 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:37.972561 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e251c2-a02b-41a8-bdaa-96dcb7016e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:37.972850 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e251c2-a02b-41a8-bdaa-96dcb7016e76 None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:37.973511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e251c2-a02b-41a8-bdaa-96dcb7016e76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:37.973813 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e251c2-a02b-41a8-bdaa-96dcb7016e76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:37.975231 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e251c2-a02b-41a8-bdaa-96dcb7016e76 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:37.986549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 624/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:18:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:37.992389 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd3bd1-8216-4c2a-b0b4-81533f0311b3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:38.083119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:38.092268 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd3bd1-8216-4c2a-b0b4-81533f0311b3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:38.124479 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45efa7-77f9-40fc-a94f-2865de7fffc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:38.125116 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45efa7-77f9-40fc-a94f-2865de7fffc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:38.125391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45efa7-77f9-40fc-a94f-2865de7fffc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:38.125660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45efa7-77f9-40fc-a94f-2865de7fffc2 None admin] PATH_INFO: `/v3/projects/32bbb936501846e8914acf07890c2891/users/2b3b0a0317b14e29a32eb6ba6b89c2aa/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:38.127598 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45efa7-77f9-40fc-a94f-2865de7fffc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=32bbb936501846e8914acf07890c2891, user_id=2b3b0a0317b14e29a32eb6ba6b89c2aa, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:38.129022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45efa7-77f9-40fc-a94f-2865de7fffc2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:38.141683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 649/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Sep 12 23:18:37 2022] PUT /identity/v3/projects/32bbb936501846e8914acf07890c2891/users/2b3b0a0317b14e29a32eb6ba6b89c2aa/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:18:40.250189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3b414-04cb-41b7-a0db-aef73ee4f3cd None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:40.250685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3b414-04cb-41b7-a0db-aef73ee4f3cd None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:40.250685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3b414-04cb-41b7-a0db-aef73ee4f3cd None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:40.252033 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 625/1274] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:18:40.257159 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e6f69-464b-4907-a7cb-265bce094d34 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:40.257583 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e6f69-464b-4907-a7cb-265bce094d34 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:40.257704 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e6f69-464b-4907-a7cb-265bce094d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:40.275341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-311e6f69-464b-4907-a7cb-265bce094d34 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:18:40.276542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-311e6f69-464b-4907-a7cb-265bce094d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:18:40.344826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 650/1275] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:18:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:18:40.621418 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3b414-04cb-41b7-a0db-aef73ee4f3cd None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:40.731433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3b414-04cb-41b7-a0db-aef73ee4f3cd None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:40.748860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:40.765218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021da52a-d374-431b-94a8-f911e052e404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:40.765834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021da52a-d374-431b-94a8-f911e052e404 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:40.766111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021da52a-d374-431b-94a8-f911e052e404 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:40.766383 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021da52a-d374-431b-94a8-f911e052e404 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:40.775998 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021da52a-d374-431b-94a8-f911e052e404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-021da52a-d374-431b-94a8-f911e052e404 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:18:40.781026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:40.782123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:40.783168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:40.783168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:40.783168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:40.783168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:40.783168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:18:40.783168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:18:40.783168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:18:40.783168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:18:40.783168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:18:40.783168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:18:40.783168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:18:40.783168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:40.783168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 626/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Sep 12 23:18:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:40.788234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e6f69-464b-4907-a7cb-265bce094d34 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:40.865785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:40.884146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e6f69-464b-4907-a7cb-265bce094d34 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:40.916325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279db77-94d7-49e7-9456-2c588fe18284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:40.916926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279db77-94d7-49e7-9456-2c588fe18284 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:40.917204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279db77-94d7-49e7-9456-2c588fe18284 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:40.917479 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279db77-94d7-49e7-9456-2c588fe18284 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:40.918330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279db77-94d7-49e7-9456-2c588fe18284 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:40.918428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279db77-94d7-49e7-9456-2c588fe18284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:40.920312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279db77-94d7-49e7-9456-2c588fe18284 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:40.928348 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 651/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Sep 12 23:18:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:40.934113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021da52a-d374-431b-94a8-f911e052e404 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:41.030187 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021da52a-d374-431b-94a8-f911e052e404 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:41.062320 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7552a-7471-47fa-b8c4-4691420097b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:41.062923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7552a-7471-47fa-b8c4-4691420097b5 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:41.063197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7552a-7471-47fa-b8c4-4691420097b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:41.063465 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7552a-7471-47fa-b8c4-4691420097b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:41.072960 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7552a-7471-47fa-b8c4-4691420097b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:41.074385 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7552a-7471-47fa-b8c4-4691420097b5 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-55a7552a-7471-47fa-b8c4-4691420097b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:18:41.088189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:41.089424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:18:41.090663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:18:41.090663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:41.090663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:41.090663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 627/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:41.095284 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279db77-94d7-49e7-9456-2c588fe18284 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:41.198892 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279db77-94d7-49e7-9456-2c588fe18284 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:41.230874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6bfad-b4ad-410b-9fde-b59543604820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:41.231470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6bfad-b4ad-410b-9fde-b59543604820 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:41.231775 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6bfad-b4ad-410b-9fde-b59543604820 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:41.232052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6bfad-b4ad-410b-9fde-b59543604820 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:41.232705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6bfad-b4ad-410b-9fde-b59543604820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:41.233010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6bfad-b4ad-410b-9fde-b59543604820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:41.234363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6bfad-b4ad-410b-9fde-b59543604820 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:41.245525 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 652/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:41.251642 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7552a-7471-47fa-b8c4-4691420097b5 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:41.281244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:41.346951 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7552a-7471-47fa-b8c4-4691420097b5 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:41.378240 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4dfd6-028a-4da2-bd60-8412d2594767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:41.378848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4dfd6-028a-4da2-bd60-8412d2594767 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:41.379129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4dfd6-028a-4da2-bd60-8412d2594767 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:41.379397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4dfd6-028a-4da2-bd60-8412d2594767 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:41.386582 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4dfd6-028a-4da2-bd60-8412d2594767 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:41.388035 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4dfd6-028a-4da2-bd60-8412d2594767 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-d7a4dfd6-028a-4da2-bd60-8412d2594767 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:18:41.399197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:41.400355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:41.401522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:41.401522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:41.401522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:18:41.401522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:18:41.401522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:18:41.401522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:18:41.401522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:18:41.401522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:18:41.401522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:18:41.401522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:41.401522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 628/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:41.406760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6bfad-b4ad-410b-9fde-b59543604820 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:41.410441 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:41.502187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6bfad-b4ad-410b-9fde-b59543604820 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:41.535198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44ef22-337f-4eac-ae6a-a836c363f666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:41.535814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44ef22-337f-4eac-ae6a-a836c363f666 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:41.536100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44ef22-337f-4eac-ae6a-a836c363f666 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:41.536372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44ef22-337f-4eac-ae6a-a836c363f666 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:41.537081 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44ef22-337f-4eac-ae6a-a836c363f666 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:41.537389 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44ef22-337f-4eac-ae6a-a836c363f666 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:41.538794 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44ef22-337f-4eac-ae6a-a836c363f666 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:41.554771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 653/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Sep 12 23:18:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:41.560499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4dfd6-028a-4da2-bd60-8412d2594767 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:41.649780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:41.660463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4dfd6-028a-4da2-bd60-8412d2594767 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:41.692541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac7bca-3f68-4999-b929-ee6a9bb87de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:41.693139 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac7bca-3f68-4999-b929-ee6a9bb87de0 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:41.693420 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac7bca-3f68-4999-b929-ee6a9bb87de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:41.693695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac7bca-3f68-4999-b929-ee6a9bb87de0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:41.758932 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac7bca-3f68-4999-b929-ee6a9bb87de0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:41.760344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac7bca-3f68-4999-b929-ee6a9bb87de0 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:41.844681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:41.854948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 629/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Sep 12 23:18:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:18:41.861390 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44ef22-337f-4eac-ae6a-a836c363f666 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:41.921527 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:41.959582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44ef22-337f-4eac-ae6a-a836c363f666 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:41.991517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183afa3-d257-434e-96ec-f3b2f4225d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:41.992162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183afa3-d257-434e-96ec-f3b2f4225d29 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:41.992441 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183afa3-d257-434e-96ec-f3b2f4225d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:41.992712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183afa3-d257-434e-96ec-f3b2f4225d29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:42.002729 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0183afa3-d257-434e-96ec-f3b2f4225d29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:42.004198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0183afa3-d257-434e-96ec-f3b2f4225d29 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-0183afa3-d257-434e-96ec-f3b2f4225d29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:18:42.018280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:42.019362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:18:42.020486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:18:42.020486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:18:42.020486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:42.020486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 654/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:42.024940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac7bca-3f68-4999-b929-ee6a9bb87de0 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:42.119669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac7bca-3f68-4999-b929-ee6a9bb87de0 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:42.152663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b05ed-a95c-491e-894f-eb643a663cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:42.153246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b05ed-a95c-491e-894f-eb643a663cbd None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:42.153525 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b05ed-a95c-491e-894f-eb643a663cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:42.153796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b05ed-a95c-491e-894f-eb643a663cbd None admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:42.155252 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b05ed-a95c-491e-894f-eb643a663cbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:42.155555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b05ed-a95c-491e-894f-eb643a663cbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:42.156946 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b05ed-a95c-491e-894f-eb643a663cbd None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:42.167698 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 630/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Sep 12 23:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:42.173234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183afa3-d257-434e-96ec-f3b2f4225d29 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:42.272466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183afa3-d257-434e-96ec-f3b2f4225d29 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:42.286087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:42.305446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04349f-d34c-4726-96d9-1d931323c21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:42.306058 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04349f-d34c-4726-96d9-1d931323c21c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:42.306339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04349f-d34c-4726-96d9-1d931323c21c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:42.306618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04349f-d34c-4726-96d9-1d931323c21c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:42.312965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04349f-d34c-4726-96d9-1d931323c21c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:42.314383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04349f-d34c-4726-96d9-1d931323c21c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5d04349f-d34c-4726-96d9-1d931323c21c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:18:42.325341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:42.326882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:42.328224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:18:42.328224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:18:42.328224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:18:42.328224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:18:42.328224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:18:42.328224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:18:42.328224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:18:42.328224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 12 23:18:42.328224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 655/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Sep 12 23:18:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:42.332728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b05ed-a95c-491e-894f-eb643a663cbd None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:42.387923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:42.436307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b05ed-a95c-491e-894f-eb643a663cbd None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:42.468433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8722580-aaea-4f95-bdc2-dc60e48c00b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:42.469025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8722580-aaea-4f95-bdc2-dc60e48c00b2 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:42.469301 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8722580-aaea-4f95-bdc2-dc60e48c00b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:42.469597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8722580-aaea-4f95-bdc2-dc60e48c00b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:42.470271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8722580-aaea-4f95-bdc2-dc60e48c00b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:42.470570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8722580-aaea-4f95-bdc2-dc60e48c00b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:42.472033 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8722580-aaea-4f95-bdc2-dc60e48c00b2 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:42.483287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 631/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Sep 12 23:18:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:42.489275 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04349f-d34c-4726-96d9-1d931323c21c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:42.585788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04349f-d34c-4726-96d9-1d931323c21c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:42.618820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f14a7c-52d1-4bde-800a-4ab054b6e8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:42.619404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f14a7c-52d1-4bde-800a-4ab054b6e8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:42.619713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f14a7c-52d1-4bde-800a-4ab054b6e8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:42.619986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f14a7c-52d1-4bde-800a-4ab054b6e8f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:42.620702 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f14a7c-52d1-4bde-800a-4ab054b6e8f4 None admin] RBAC: Adding query filter params (role.id=23538dc4c770454e80dd11ed8e498539, scope.project.id=32bbb936501846e8914acf07890c2891, user.id=2b3b0a0317b14e29a32eb6ba6b89c2aa) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:42.621008 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f14a7c-52d1-4bde-800a-4ab054b6e8f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:42.622412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f14a7c-52d1-4bde-800a-4ab054b6e8f4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:42.628493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 656/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Sep 12 23:18:42 2022] GET /identity/v3/role_assignments?user.id=2b3b0a0317b14e29a32eb6ba6b89c2aa&scope.project.id=32bbb936501846e8914acf07890c2891&role.id=23538dc4c770454e80dd11ed8e498539 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:44.764369 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b780060-a103-418d-8ecc-31202ecb96a6 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:44.764902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b780060-a103-418d-8ecc-31202ecb96a6 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:44.764902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b780060-a103-418d-8ecc-31202ecb96a6 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:44.766250 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 632/1288] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:18:44.771258 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af824289-24a2-4d61-96a7-f0b189663b56 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:44.771670 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af824289-24a2-4d61-96a7-f0b189663b56 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:44.771830 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af824289-24a2-4d61-96a7-f0b189663b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:44.790290 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-af824289-24a2-4d61-96a7-f0b189663b56 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:18:44.791457 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-af824289-24a2-4d61-96a7-f0b189663b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:18:44.819855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:44.855895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 657/1289] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:18:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:18:45.118902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b780060-a103-418d-8ecc-31202ecb96a6 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:45.216115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b780060-a103-418d-8ecc-31202ecb96a6 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:45.226284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:45.250805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f90b73-d5ba-4dac-8468-027d566a60c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:45.251429 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f90b73-d5ba-4dac-8468-027d566a60c2 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:45.251741 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f90b73-d5ba-4dac-8468-027d566a60c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:45.252018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f90b73-d5ba-4dac-8468-027d566a60c2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:45.252593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f90b73-d5ba-4dac-8468-027d566a60c2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:45.253962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f90b73-d5ba-4dac-8468-027d566a60c2 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-77f90b73-d5ba-4dac-8468-027d566a60c2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:45.262555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:45.263838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:45.264888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 633/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Sep 12 23:18:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:45.269514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af824289-24a2-4d61-96a7-f0b189663b56 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:45.370444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af824289-24a2-4d61-96a7-f0b189663b56 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:45.402920 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4904085-f262-410e-97a5-325ed4e7170b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:45.403522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4904085-f262-410e-97a5-325ed4e7170b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:45.404097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4904085-f262-410e-97a5-325ed4e7170b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:45.404378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4904085-f262-410e-97a5-325ed4e7170b None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:45.405005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4904085-f262-410e-97a5-325ed4e7170b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:45.405313 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4904085-f262-410e-97a5-325ed4e7170b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:45.406671 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4904085-f262-410e-97a5-325ed4e7170b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:45.411257 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 658/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Sep 12 23:18:45 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:45.417191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f90b73-d5ba-4dac-8468-027d566a60c2 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:45.513763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f90b73-d5ba-4dac-8468-027d566a60c2 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:45.546543 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653adb4-4107-4170-ad20-979975f7c521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:45.547126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653adb4-4107-4170-ad20-979975f7c521 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:45.547409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653adb4-4107-4170-ad20-979975f7c521 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:45.547715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653adb4-4107-4170-ad20-979975f7c521 None admin] PATH_INFO: `/v3/services` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:45.548332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653adb4-4107-4170-ad20-979975f7c521 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:45.548635 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653adb4-4107-4170-ad20-979975f7c521 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:45.549989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653adb4-4107-4170-ad20-979975f7c521 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:45.554232 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 634/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Sep 12 23:18:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:47.690430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a0136-27a5-4e15-9a31-503cdd963adc None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:47.690997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a0136-27a5-4e15-9a31-503cdd963adc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:47.690997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a0136-27a5-4e15-9a31-503cdd963adc None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:47.692350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 659/1293] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:18:47.697510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b25bbb-8132-404f-87fc-2d9a696ec27a None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:47.697878 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b25bbb-8132-404f-87fc-2d9a696ec27a None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:47.698059 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b25bbb-8132-404f-87fc-2d9a696ec27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:47.716992 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-04b25bbb-8132-404f-87fc-2d9a696ec27a None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:18:47.718171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-04b25bbb-8132-404f-87fc-2d9a696ec27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:18:47.746197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:47.784365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 635/1294] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:18:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:18:48.061522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a0136-27a5-4e15-9a31-503cdd963adc None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:48.092138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:48.163312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a0136-27a5-4e15-9a31-503cdd963adc None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:48.198806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a1a4d-5137-43da-a432-648ad048052e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:48.199405 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a1a4d-5137-43da-a432-648ad048052e None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:48.199712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a1a4d-5137-43da-a432-648ad048052e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:48.199986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a1a4d-5137-43da-a432-648ad048052e None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:48.200649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a1a4d-5137-43da-a432-648ad048052e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:48.201999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a1a4d-5137-43da-a432-648ad048052e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:48.215516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 660/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Sep 12 23:18:48 2022] POST /identity/v3/services => generated 246 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:18:50.338022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777660fa-de1a-4016-946c-73696158f27a None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:50.338525 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777660fa-de1a-4016-946c-73696158f27a None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:50.338525 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777660fa-de1a-4016-946c-73696158f27a None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:50.339882 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 636/1296] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:18:50.344991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b293242-ea4b-488f-af3a-db9d5de1262f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:50.345406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b293242-ea4b-488f-af3a-db9d5de1262f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:50.345543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b293242-ea4b-488f-af3a-db9d5de1262f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:50.363698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1b293242-ea4b-488f-af3a-db9d5de1262f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:18:50.364883 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1b293242-ea4b-488f-af3a-db9d5de1262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:18:50.434598 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 661/1297] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:18:50 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:18:50.723468 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777660fa-de1a-4016-946c-73696158f27a None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:50.822160 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777660fa-de1a-4016-946c-73696158f27a None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:50.854629 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de3f67-8e84-4755-bb17-ddfb868da884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:50.855239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de3f67-8e84-4755-bb17-ddfb868da884 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:50.855518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de3f67-8e84-4755-bb17-ddfb868da884 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:50.855813 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de3f67-8e84-4755-bb17-ddfb868da884 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:50.856381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de3f67-8e84-4755-bb17-ddfb868da884 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:50.857838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de3f67-8e84-4755-bb17-ddfb868da884 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-67de3f67-8e84-4755-bb17-ddfb868da884 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:50.866299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:50.867511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:50.868397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 637/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Sep 12 23:18:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:50.873799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b293242-ea4b-488f-af3a-db9d5de1262f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:50.891931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:50.969241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b293242-ea4b-488f-af3a-db9d5de1262f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:51.001294 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443321f-abf3-44be-a8c7-8a87f6f7e9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:51.001830 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443321f-abf3-44be-a8c7-8a87f6f7e9ac None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:51.002106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443321f-abf3-44be-a8c7-8a87f6f7e9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:51.002375 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443321f-abf3-44be-a8c7-8a87f6f7e9ac None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:51.002998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443321f-abf3-44be-a8c7-8a87f6f7e9ac None admin] RBAC: Adding query filter params (name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:51.003296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443321f-abf3-44be-a8c7-8a87f6f7e9ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:51.004674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443321f-abf3-44be-a8c7-8a87f6f7e9ac None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:51.009504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 662/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Sep 12 23:18:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:51.015974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de3f67-8e84-4755-bb17-ddfb868da884 None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:51.046538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:51.112199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de3f67-8e84-4755-bb17-ddfb868da884 None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:51.143814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e24893-c1d0-49f8-841d-eb37a27e31c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:51.144391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24893-c1d0-49f8-841d-eb37a27e31c0 None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:51.144667 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24893-c1d0-49f8-841d-eb37a27e31c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:51.144938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24893-c1d0-49f8-841d-eb37a27e31c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:51.145616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e24893-c1d0-49f8-841d-eb37a27e31c0 None admin] RBAC: Adding query filter params (interface=public, service_id=4dc9615ec1a5459e8c06d74b57cd77ac, region_id=RegionOne) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:51.145918 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e24893-c1d0-49f8-841d-eb37a27e31c0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:51.147265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e24893-c1d0-49f8-841d-eb37a27e31c0 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:51.152005 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 638/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Sep 12 23:18:51 2022] GET /identity/v3/endpoints?service_id=4dc9615ec1a5459e8c06d74b57cd77ac&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:51.157816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443321f-abf3-44be-a8c7-8a87f6f7e9ac None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:51.235881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:51.253696 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443321f-abf3-44be-a8c7-8a87f6f7e9ac None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:51.285653 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a30f5-37c2-4491-a35d-2ae48d08f0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:51.286243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a30f5-37c2-4491-a35d-2ae48d08f0ab None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:51.286522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a30f5-37c2-4491-a35d-2ae48d08f0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:51.286796 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a30f5-37c2-4491-a35d-2ae48d08f0ab None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:51.287389 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a30f5-37c2-4491-a35d-2ae48d08f0ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:51.288762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a30f5-37c2-4491-a35d-2ae48d08f0ab None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:51.295743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 663/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Sep 12 23:18:51 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:18:53.409680 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea90a44-01a2-4d84-a358-3dae0b205a28 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:53.410175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea90a44-01a2-4d84-a358-3dae0b205a28 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:53.410175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea90a44-01a2-4d84-a358-3dae0b205a28 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:53.411487 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 639/1302] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:18:53.416687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8210c7-9f57-4afd-9d9e-f63d0e8472b2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:53.417076 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8210c7-9f57-4afd-9d9e-f63d0e8472b2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:53.417241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8210c7-9f57-4afd-9d9e-f63d0e8472b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:53.434820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5d8210c7-9f57-4afd-9d9e-f63d0e8472b2 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:18:53.436048 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5d8210c7-9f57-4afd-9d9e-f63d0e8472b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:18:53.499762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 664/1303] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:18:53 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:18:53.770981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea90a44-01a2-4d84-a358-3dae0b205a28 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:53.864219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:53.870886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea90a44-01a2-4d84-a358-3dae0b205a28 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:53.904538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e86a6-08f8-4cf4-a1c7-e9d7c91d948d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:53.905157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e86a6-08f8-4cf4-a1c7-e9d7c91d948d None admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:53.905435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e86a6-08f8-4cf4-a1c7-e9d7c91d948d None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:53.905711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e86a6-08f8-4cf4-a1c7-e9d7c91d948d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:53.906283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e86a6-08f8-4cf4-a1c7-e9d7c91d948d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:53.907670 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e86a6-08f8-4cf4-a1c7-e9d7c91d948d None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application [None req-1c9e86a6-08f8-4cf4-a1c7-e9d7c91d948d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:18:53.918880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:18:53.920156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application with Lock( Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: ERROR keystone.server.flask.application Sep 12 23:18:53.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 640/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Sep 12 23:18:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:18:53.925937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8210c7-9f57-4afd-9d9e-f63d0e8472b2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:53.998330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:18:54.020184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8210c7-9f57-4afd-9d9e-f63d0e8472b2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:54.052049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e31bba-f308-43a6-b29d-6f2a41fcf4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:54.052575 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e31bba-f308-43a6-b29d-6f2a41fcf4cb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:54.052854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e31bba-f308-43a6-b29d-6f2a41fcf4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:54.053124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e31bba-f308-43a6-b29d-6f2a41fcf4cb None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:54.053735 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e31bba-f308-43a6-b29d-6f2a41fcf4cb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:18:54.054035 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e31bba-f308-43a6-b29d-6f2a41fcf4cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:54.055386 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e31bba-f308-43a6-b29d-6f2a41fcf4cb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:54.060782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 665/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Sep 12 23:18:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:18:54.066511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e86a6-08f8-4cf4-a1c7-e9d7c91d948d None admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:18:54.161973 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e86a6-08f8-4cf4-a1c7-e9d7c91d948d None admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:18:54.194860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f5c0d-4562-44ec-8c61-b050a8490136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:18:54.195443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f5c0d-4562-44ec-8c61-b050a8490136 None admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:18:54.195751 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f5c0d-4562-44ec-8c61-b050a8490136 None admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:18:54.196023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f5c0d-4562-44ec-8c61-b050a8490136 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:18:54.196678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f5c0d-4562-44ec-8c61-b050a8490136 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:18:54.198028 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f5c0d-4562-44ec-8c61-b050a8490136 None admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:18:54.217752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 641/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:18:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:19:04.057441 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=30 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:19:04.213646 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:19:58.146157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7deae3-875b-4afc-86f1-71159360d852 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:19:58.146635 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7deae3-875b-4afc-86f1-71159360d852 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:19:58.146971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7deae3-875b-4afc-86f1-71159360d852 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:19:58.152586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 666/1307] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:19:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:19:58.156014 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87825ec3-857a-46a4-8af5-2ba357caca7e None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:19:58.156476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87825ec3-857a-46a4-8af5-2ba357caca7e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:19:58.156779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87825ec3-857a-46a4-8af5-2ba357caca7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:19:58.266024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-87825ec3-857a-46a4-8af5-2ba357caca7e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:19:58.267519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-87825ec3-857a-46a4-8af5-2ba357caca7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:19:58.388312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 642/1308] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:19:58.682612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fc613-e594-4dc8-99f5-9445a07226f7 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:19:58.682890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fc613-e594-4dc8-99f5-9445a07226f7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:19:58.683333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fc613-e594-4dc8-99f5-9445a07226f7 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:19:58.684829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 667/1309] 10.222.0.27 () {54 vars in 965 bytes} [Mon Sep 12 23:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:19:58.690796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e712da5-7fea-42b7-9264-4d3e873fd869 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:19:58.691205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e712da5-7fea-42b7-9264-4d3e873fd869 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:19:58.691592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e712da5-7fea-42b7-9264-4d3e873fd869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:19:58.776997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-6e712da5-7fea-42b7-9264-4d3e873fd869 None None] MFA Rules not processed for user `bc6e63e9fc3447d5a894a777a98fe773`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:19:58.778146 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6e712da5-7fea-42b7-9264-4d3e873fd869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:19:58.800058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:19:58.868367 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fc613-e594-4dc8-99f5-9445a07226f7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:19:58.870224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2e0fc613-e594-4dc8-99f5-9445a07226f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:19:58.878697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 643/1310] 10.222.0.27 () {58 vars in 1062 bytes} [Mon Sep 12 23:19:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:19:59.002391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fc613-e594-4dc8-99f5-9445a07226f7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:19:59.088953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe63072-aa84-4a80-a7dd-b291f6edf68b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:19:59.089684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe63072-aa84-4a80-a7dd-b291f6edf68b service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:19:59.089968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe63072-aa84-4a80-a7dd-b291f6edf68b service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:19:59.090242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe63072-aa84-4a80-a7dd-b291f6edf68b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:19:59.091869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-afe63072-aa84-4a80-a7dd-b291f6edf68b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:19:59.151547 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:19:59.213322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe63072-aa84-4a80-a7dd-b291f6edf68b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:19:59.214961 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe63072-aa84-4a80-a7dd-b291f6edf68b service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:19:59.237286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 668/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:20:01.683444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1beb23-7a5c-4843-83aa-5bfcb4388d3f None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:01.684023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1beb23-7a5c-4843-83aa-5bfcb4388d3f None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:01.684023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1beb23-7a5c-4843-83aa-5bfcb4388d3f None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:01.685390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 644/1312] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:01.690718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbcf1d-3fe3-4e51-a02d-61a78540c37e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:01.691139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbcf1d-3fe3-4e51-a02d-61a78540c37e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:01.691279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbcf1d-3fe3-4e51-a02d-61a78540c37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:01.771411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d4fbcf1d-3fe3-4e51-a02d-61a78540c37e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:01.772647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d4fbcf1d-3fe3-4e51-a02d-61a78540c37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:01.870283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 669/1313] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:02.156490 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1beb23-7a5c-4843-83aa-5bfcb4388d3f None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:20:02.158020 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3d1beb23-7a5c-4843-83aa-5bfcb4388d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:02.260124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1beb23-7a5c-4843-83aa-5bfcb4388d3f None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:20:02.330408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:02.332852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa6102-cab7-4353-8c84-ee4c3742bfc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:20:02.333584 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6102-cab7-4353-8c84-ee4c3742bfc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:02.334133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6102-cab7-4353-8c84-ee4c3742bfc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:02.334656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6102-cab7-4353-8c84-ee4c3742bfc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:02.336527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-56fa6102-cab7-4353-8c84-ee4c3742bfc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:02.451692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa6102-cab7-4353-8c84-ee4c3742bfc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:20:02.453153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa6102-cab7-4353-8c84-ee4c3742bfc4 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:20:02.472783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 645/1314] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:20:04.954860 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf404d8-94bd-4bc5-b392-c760413a8ced None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:04.955315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf404d8-94bd-4bc5-b392-c760413a8ced None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:04.955424 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf404d8-94bd-4bc5-b392-c760413a8ced None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:04.956854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 670/1315] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:04.962671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bd73e-c32a-4b7d-a3d3-8e5eaf1e1908 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:04.963080 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bd73e-c32a-4b7d-a3d3-8e5eaf1e1908 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:04.963226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bd73e-c32a-4b7d-a3d3-8e5eaf1e1908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:05.029709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-8d6bd73e-c32a-4b7d-a3d3-8e5eaf1e1908 None None] MFA Rules not processed for user `28ab6d08782b4c26b0afe016bbcd8688`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:05.030910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8d6bd73e-c32a-4b7d-a3d3-8e5eaf1e1908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:05.082100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:05.125273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 646/1316] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:20:04 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:05.408811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf404d8-94bd-4bc5-b392-c760413a8ced None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:20:05.410339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-acf404d8-94bd-4bc5-b392-c760413a8ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:05.420643 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:05.512216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf404d8-94bd-4bc5-b392-c760413a8ced None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:20:05.584831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3d5b3-b02b-44b7-9939-11a6ad8f4743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:20:05.585437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3d5b3-b02b-44b7-9939-11a6ad8f4743 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:05.585713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3d5b3-b02b-44b7-9939-11a6ad8f4743 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:05.585991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3d5b3-b02b-44b7-9939-11a6ad8f4743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:05.587600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a0e3d5b3-b02b-44b7-9939-11a6ad8f4743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:05.649698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:05.699285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3d5b3-b02b-44b7-9939-11a6ad8f4743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:20:05.701426 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3d5b3-b02b-44b7-9939-11a6ad8f4743 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:20:05.724945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 671/1317] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:20:05 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:20:08.390322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251b559-cd30-4deb-9b44-66283c23c19e None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:08.390856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251b559-cd30-4deb-9b44-66283c23c19e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:08.390856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251b559-cd30-4deb-9b44-66283c23c19e None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:08.392154 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 647/1318] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:08.397217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433cc25-9a6b-4cba-bd18-8fdd1382552b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:08.397618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433cc25-9a6b-4cba-bd18-8fdd1382552b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:08.397764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433cc25-9a6b-4cba-bd18-8fdd1382552b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:08.463842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a433cc25-9a6b-4cba-bd18-8fdd1382552b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:08.465074 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a433cc25-9a6b-4cba-bd18-8fdd1382552b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:08.558586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 672/1319] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:20:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:08.840622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251b559-cd30-4deb-9b44-66283c23c19e None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:20:08.842186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b251b559-cd30-4deb-9b44-66283c23c19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:08.943522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251b559-cd30-4deb-9b44-66283c23c19e None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:20:09.014825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96c13b-a3ed-47a4-9a03-dcbbd911c5a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:20:09.015422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96c13b-a3ed-47a4-9a03-dcbbd911c5a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:09.015729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96c13b-a3ed-47a4-9a03-dcbbd911c5a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:09.015999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96c13b-a3ed-47a4-9a03-dcbbd911c5a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:09.017561 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5a96c13b-a3ed-47a4-9a03-dcbbd911c5a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:09.024678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:09.123151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96c13b-a3ed-47a4-9a03-dcbbd911c5a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:20:09.124535 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96c13b-a3ed-47a4-9a03-dcbbd911c5a1 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:20:09.142303 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 648/1320] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:20:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:20:11.897481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a95c3-e069-4362-af46-3856bf186768 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:11.897991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a95c3-e069-4362-af46-3856bf186768 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:11.897991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a95c3-e069-4362-af46-3856bf186768 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:11.899833 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 673/1321] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:11.905264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270c885-2192-446d-8843-881e63fd6a0c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:11.905670 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270c885-2192-446d-8843-881e63fd6a0c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:11.905953 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270c885-2192-446d-8843-881e63fd6a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:11.981439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b270c885-2192-446d-8843-881e63fd6a0c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:11.982684 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b270c885-2192-446d-8843-881e63fd6a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:12.044349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:12.086095 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 649/1322] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:12.368570 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a95c3-e069-4362-af46-3856bf186768 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:20:12.370088 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e79a95c3-e069-4362-af46-3856bf186768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:12.389756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:12.479330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a95c3-e069-4362-af46-3856bf186768 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:20:12.555829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2918df5-d19d-441e-9e6b-3f26cf566149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:20:12.556435 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2918df5-d19d-441e-9e6b-3f26cf566149 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:12.556738 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2918df5-d19d-441e-9e6b-3f26cf566149 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:12.557013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2918df5-d19d-441e-9e6b-3f26cf566149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:12.558604 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f2918df5-d19d-441e-9e6b-3f26cf566149 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:12.630553 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:12.670003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2918df5-d19d-441e-9e6b-3f26cf566149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:20:12.671436 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2918df5-d19d-441e-9e6b-3f26cf566149 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:20:12.690168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 674/1323] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:20:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:20:14.915503 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f845178-17b0-4d29-9b66-7c9979446d9b None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:14.916057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f845178-17b0-4d29-9b66-7c9979446d9b None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:14.916057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f845178-17b0-4d29-9b66-7c9979446d9b None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:14.917552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 650/1324] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:14.922623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e31560-c8d9-4343-ad45-52be848e0723 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:14.923031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e31560-c8d9-4343-ad45-52be848e0723 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:14.923180 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e31560-c8d9-4343-ad45-52be848e0723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:14.988954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-58e31560-c8d9-4343-ad45-52be848e0723 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:14.990140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-58e31560-c8d9-4343-ad45-52be848e0723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:15.099188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 675/1325] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:15.380373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f845178-17b0-4d29-9b66-7c9979446d9b None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:20:15.381895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9f845178-17b0-4d29-9b66-7c9979446d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:15.497873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f845178-17b0-4d29-9b66-7c9979446d9b None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:20:15.569526 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610c65e-0062-49bc-b003-d26668ce4726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:20:15.570728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610c65e-0062-49bc-b003-d26668ce4726 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:15.571001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610c65e-0062-49bc-b003-d26668ce4726 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:15.571264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610c65e-0062-49bc-b003-d26668ce4726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:15.573374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9610c65e-0062-49bc-b003-d26668ce4726 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:15.589985 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:15.682877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9610c65e-0062-49bc-b003-d26668ce4726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:20:15.684475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9610c65e-0062-49bc-b003-d26668ce4726 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:20:15.702499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 651/1326] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:20:17.898860 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ee8e0-98e5-4abe-8f95-7c7234a6ba69 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:17.899356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ee8e0-98e5-4abe-8f95-7c7234a6ba69 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:17.899460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ee8e0-98e5-4abe-8f95-7c7234a6ba69 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:17.900754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 676/1327] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:17.906209 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38083e6-d12c-4882-9249-bafd113be32d None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:17.906622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38083e6-d12c-4882-9249-bafd113be32d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:17.906761 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38083e6-d12c-4882-9249-bafd113be32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:17.984349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b38083e6-d12c-4882-9249-bafd113be32d None None] MFA Rules not processed for user `28ab6d08782b4c26b0afe016bbcd8688`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:17.985931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b38083e6-d12c-4882-9249-bafd113be32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:18.051554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:18.083660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 652/1328] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:20:17 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:18.366470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ee8e0-98e5-4abe-8f95-7c7234a6ba69 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:20:18.368037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-861ee8e0-98e5-4abe-8f95-7c7234a6ba69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:18.401425 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:18.474673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ee8e0-98e5-4abe-8f95-7c7234a6ba69 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:20:18.549874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06787423-e356-4eb6-aaff-572ae787a412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:20:18.550484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06787423-e356-4eb6-aaff-572ae787a412 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:18.550764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06787423-e356-4eb6-aaff-572ae787a412 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:18.551037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06787423-e356-4eb6-aaff-572ae787a412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:18.552649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-06787423-e356-4eb6-aaff-572ae787a412 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:18.630030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:18.661940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06787423-e356-4eb6-aaff-572ae787a412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:20:18.663392 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06787423-e356-4eb6-aaff-572ae787a412 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:20:18.681453 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 677/1329] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:20:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:20:21.049281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecf391-c049-40b2-b889-cc6643d58597 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:21.049833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecf391-c049-40b2-b889-cc6643d58597 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:21.049833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecf391-c049-40b2-b889-cc6643d58597 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:21.051152 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 653/1330] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:21.056872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716892a-b3c6-4492-b7df-98ea0d7fe4b0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:21.057272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716892a-b3c6-4492-b7df-98ea0d7fe4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:21.057414 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716892a-b3c6-4492-b7df-98ea0d7fe4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:21.121592 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0716892a-b3c6-4492-b7df-98ea0d7fe4b0 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:21.122777 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0716892a-b3c6-4492-b7df-98ea0d7fe4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:21.222155 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 678/1331] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:21.499404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecf391-c049-40b2-b889-cc6643d58597 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:20:21.500971 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-27ecf391-c049-40b2-b889-cc6643d58597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:21.611258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecf391-c049-40b2-b889-cc6643d58597 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:20:21.682851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb375e2-7df8-402d-825f-61ea25b0a96c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:20:21.683454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb375e2-7df8-402d-825f-61ea25b0a96c service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:21.683768 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb375e2-7df8-402d-825f-61ea25b0a96c service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:21.684042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb375e2-7df8-402d-825f-61ea25b0a96c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:21.685624 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2bb375e2-7df8-402d-825f-61ea25b0a96c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:21.711755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:21.791968 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb375e2-7df8-402d-825f-61ea25b0a96c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:20:21.793342 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb375e2-7df8-402d-825f-61ea25b0a96c service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:20:21.811207 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 654/1332] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:20:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:20:24.039007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3e98d-f2f7-4a3f-ad9f-0c3d3ef830a9 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:24.039527 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3e98d-f2f7-4a3f-ad9f-0c3d3ef830a9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:24.039665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3e98d-f2f7-4a3f-ad9f-0c3d3ef830a9 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:24.040927 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 679/1333] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:24.046189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd0571-c3b3-4625-8cb8-b144e4c91581 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:24.046559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd0571-c3b3-4625-8cb8-b144e4c91581 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:24.046745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd0571-c3b3-4625-8cb8-b144e4c91581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:24.111054 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-29fd0571-c3b3-4625-8cb8-b144e4c91581 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:24.112275 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-29fd0571-c3b3-4625-8cb8-b144e4c91581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:24.184868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:24.207908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 655/1334] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:24.492987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3e98d-f2f7-4a3f-ad9f-0c3d3ef830a9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:20:24.494568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-faa3e98d-f2f7-4a3f-ad9f-0c3d3ef830a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:24.534545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:24.598789 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3e98d-f2f7-4a3f-ad9f-0c3d3ef830a9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:20:24.671737 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7291a-36e5-4b10-a63e-2e0276e3f1ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:20:24.672337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7291a-36e5-4b10-a63e-2e0276e3f1ea service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:24.672616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7291a-36e5-4b10-a63e-2e0276e3f1ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:24.672890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7291a-36e5-4b10-a63e-2e0276e3f1ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:24.674464 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8dc7291a-36e5-4b10-a63e-2e0276e3f1ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:24.755420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:24.780896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7291a-36e5-4b10-a63e-2e0276e3f1ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:20:24.782325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7291a-36e5-4b10-a63e-2e0276e3f1ea service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:20:24.800174 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 680/1335] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:20:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:20:27.270950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df829995-c72a-4b7d-bfae-16c57b2b7dc2 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:27.271428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df829995-c72a-4b7d-bfae-16c57b2b7dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:27.271538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df829995-c72a-4b7d-bfae-16c57b2b7dc2 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:27.272902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 656/1336] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:27.278311 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ac31f-2d8a-4d57-ba04-cbf5265a2619 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:27.278765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ac31f-2d8a-4d57-ba04-cbf5265a2619 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:27.278878 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ac31f-2d8a-4d57-ba04-cbf5265a2619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:27.345253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b07ac31f-2d8a-4d57-ba04-cbf5265a2619 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:27.346458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b07ac31f-2d8a-4d57-ba04-cbf5265a2619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:27.440458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 681/1337] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:20:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:27.719350 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df829995-c72a-4b7d-bfae-16c57b2b7dc2 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:20:27.720910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-df829995-c72a-4b7d-bfae-16c57b2b7dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:27.821962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df829995-c72a-4b7d-bfae-16c57b2b7dc2 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:20:27.892381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3fad8-f396-4672-be99-0e9b9f90a7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:20:27.892914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3fad8-f396-4672-be99-0e9b9f90a7ea service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:27.893193 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3fad8-f396-4672-be99-0e9b9f90a7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:27.893462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3fad8-f396-4672-be99-0e9b9f90a7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:27.895024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-60d3fad8-f396-4672-be99-0e9b9f90a7ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:27.931334 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:28.009689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d3fad8-f396-4672-be99-0e9b9f90a7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:20:28.011125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d3fad8-f396-4672-be99-0e9b9f90a7ea service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:20:28.029022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 657/1338] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:20:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:20:32.074950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2f73-1417-4bf7-aad8-bd6d15ab4d18 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:32.075504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2f73-1417-4bf7-aad8-bd6d15ab4d18 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:32.075504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2f73-1417-4bf7-aad8-bd6d15ab4d18 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:32.076844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 682/1339] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:32.082361 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b118c54-fb99-4ca0-ba3d-26e118ce7fe7 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:32.082769 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b118c54-fb99-4ca0-ba3d-26e118ce7fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:32.083006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b118c54-fb99-4ca0-ba3d-26e118ce7fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:32.148229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-1b118c54-fb99-4ca0-ba3d-26e118ce7fe7 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:32.149369 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1b118c54-fb99-4ca0-ba3d-26e118ce7fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:32.234058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:32.245909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 658/1340] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:20:32.542042 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213ce34-22bd-4941-a0ae-eeae021d210e None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:32.542431 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213ce34-22bd-4941-a0ae-eeae021d210e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:32.542801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213ce34-22bd-4941-a0ae-eeae021d210e None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:32.544270 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 683/1341] 10.222.0.27 () {54 vars in 965 bytes} [Mon Sep 12 23:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:32.549992 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23ab07-d34f-46b5-8a9c-b626cec8fc47 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:32.550372 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23ab07-d34f-46b5-8a9c-b626cec8fc47 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:32.550654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23ab07-d34f-46b5-8a9c-b626cec8fc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:32.629854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-3c23ab07-d34f-46b5-8a9c-b626cec8fc47 None None] MFA Rules not processed for user `bc6e63e9fc3447d5a894a777a98fe773`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:32.631096 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3c23ab07-d34f-46b5-8a9c-b626cec8fc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:32.705817 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 659/1342] 10.222.0.27 () {58 vars in 1062 bytes} [Mon Sep 12 23:20:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:32.715384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213ce34-22bd-4941-a0ae-eeae021d210e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:20:32.716939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8213ce34-22bd-4941-a0ae-eeae021d210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:32.766106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:32.821464 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213ce34-22bd-4941-a0ae-eeae021d210e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:20:32.901239 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38669432-5329-4388-805a-75ce187fe3f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:20:32.901846 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38669432-5329-4388-805a-75ce187fe3f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:32.902129 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38669432-5329-4388-805a-75ce187fe3f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:32.902422 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38669432-5329-4388-805a-75ce187fe3f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:32.904032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-38669432-5329-4388-805a-75ce187fe3f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:33.008156 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:33.027234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38669432-5329-4388-805a-75ce187fe3f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:20:33.028766 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38669432-5329-4388-805a-75ce187fe3f4 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:20:33.048201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 684/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:20:36.382669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6617a6a-fbb9-4353-9d46-55e552f3e8c0 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:36.383183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6617a6a-fbb9-4353-9d46-55e552f3e8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:36.383183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6617a6a-fbb9-4353-9d46-55e552f3e8c0 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:36.384527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 660/1344] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:36.390193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb3227-2b2e-480d-9c80-539659e6c4cf None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:36.390671 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb3227-2b2e-480d-9c80-539659e6c4cf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:36.390782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb3227-2b2e-480d-9c80-539659e6c4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:36.458441 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-20fb3227-2b2e-480d-9c80-539659e6c4cf None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:36.460192 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-20fb3227-2b2e-480d-9c80-539659e6c4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:36.575059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 685/1345] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:20:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:36.870042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6617a6a-fbb9-4353-9d46-55e552f3e8c0 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:20:36.871615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d6617a6a-fbb9-4353-9d46-55e552f3e8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:36.980225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:36.997330 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6617a6a-fbb9-4353-9d46-55e552f3e8c0 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:20:37.071133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e787d14-cb1b-4370-9d6a-8fde899849d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:20:37.071780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e787d14-cb1b-4370-9d6a-8fde899849d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:37.072060 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e787d14-cb1b-4370-9d6a-8fde899849d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:37.072330 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e787d14-cb1b-4370-9d6a-8fde899849d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:37.073911 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5e787d14-cb1b-4370-9d6a-8fde899849d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:37.181812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e787d14-cb1b-4370-9d6a-8fde899849d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:20:37.183265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e787d14-cb1b-4370-9d6a-8fde899849d0 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:20:37.201353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 661/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:20:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:20:41.528017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffffc8-ae1d-46da-850c-51f1ff1926a9 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:41.528449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffffc8-ae1d-46da-850c-51f1ff1926a9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:41.528566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffffc8-ae1d-46da-850c-51f1ff1926a9 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:41.529907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 686/1347] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:41.535470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56553d-c7e3-4d14-9878-4be2fde4ba06 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:41.536030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56553d-c7e3-4d14-9878-4be2fde4ba06 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:41.536124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56553d-c7e3-4d14-9878-4be2fde4ba06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:41.552144 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:41.608703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-8d56553d-c7e3-4d14-9878-4be2fde4ba06 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:41.609944 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8d56553d-c7e3-4d14-9878-4be2fde4ba06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:41.712619 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 662/1348] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:41.996500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffffc8-ae1d-46da-850c-51f1ff1926a9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:20:41.998039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f6ffffc8-ae1d-46da-850c-51f1ff1926a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:42.056817 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:42.102435 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffffc8-ae1d-46da-850c-51f1ff1926a9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:20:42.176235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae8ba8-b237-440c-9f7f-5d60b3918da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:20:42.176771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae8ba8-b237-440c-9f7f-5d60b3918da7 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:42.177050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae8ba8-b237-440c-9f7f-5d60b3918da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:42.177364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae8ba8-b237-440c-9f7f-5d60b3918da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:42.178956 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d4ae8ba8-b237-440c-9f7f-5d60b3918da7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:42.281365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:42.285489 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae8ba8-b237-440c-9f7f-5d60b3918da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:20:42.287519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae8ba8-b237-440c-9f7f-5d60b3918da7 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:20:42.305844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 687/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:20:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:20:47.213900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e58a4-7289-407e-bc2a-408ce0996983 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:47.214848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e58a4-7289-407e-bc2a-408ce0996983 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:47.215199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e58a4-7289-407e-bc2a-408ce0996983 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:47.218147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 663/1350] 10.222.0.27 () {54 vars in 921 bytes} [Mon Sep 12 23:20:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:47.222875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72d6a9-5571-441f-8f99-50c31bedd22a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:47.223269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72d6a9-5571-441f-8f99-50c31bedd22a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:47.224971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72d6a9-5571-441f-8f99-50c31bedd22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:47.351984 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bf72d6a9-5571-441f-8f99-50c31bedd22a None None] MFA Rules not processed for user `2b3b0a0317b14e29a32eb6ba6b89c2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:47.353738 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bf72d6a9-5571-441f-8f99-50c31bedd22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:47.494259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 688/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Sep 12 23:20:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:51.628729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83cd08-7ae6-4ba0-8b0a-3543b44c03b4 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:51.629228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83cd08-7ae6-4ba0-8b0a-3543b44c03b4 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:51.629341 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83cd08-7ae6-4ba0-8b0a-3543b44c03b4 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:51.630944 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 664/1352] 10.222.0.27 () {54 vars in 921 bytes} [Mon Sep 12 23:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:51.636713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4588570-d23d-4f1c-aeec-2238d43f3e74 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:51.637130 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4588570-d23d-4f1c-aeec-2238d43f3e74 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:51.637277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4588570-d23d-4f1c-aeec-2238d43f3e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:51.695631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:51.710830 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=59 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:51.719994 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a4588570-d23d-4f1c-aeec-2238d43f3e74 None None] MFA Rules not processed for user `2b3b0a0317b14e29a32eb6ba6b89c2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:51.721301 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a4588570-d23d-4f1c-aeec-2238d43f3e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:51.832865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 689/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Sep 12 23:20:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:53.232791 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366770ec-58e3-466d-a0e2-97de1f458bbd None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:53.233325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366770ec-58e3-466d-a0e2-97de1f458bbd None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:53.233325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366770ec-58e3-466d-a0e2-97de1f458bbd None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:53.234603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 665/1354] 10.222.0.27 () {54 vars in 921 bytes} [Mon Sep 12 23:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:53.240210 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc832fa6-32d1-4c44-9f9c-88a84bafb2fc None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:53.240601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc832fa6-32d1-4c44-9f9c-88a84bafb2fc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:53.240880 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc832fa6-32d1-4c44-9f9c-88a84bafb2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:53.315855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-cc832fa6-32d1-4c44-9f9c-88a84bafb2fc None None] MFA Rules not processed for user `2b3b0a0317b14e29a32eb6ba6b89c2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:53.317086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cc832fa6-32d1-4c44-9f9c-88a84bafb2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:53.358106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:53.429143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 690/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Sep 12 23:20:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:56.866079 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a05738-23e0-44ec-a1ac-faa05a7bcce9 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:56.867072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a05738-23e0-44ec-a1ac-faa05a7bcce9 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:56.867596 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a05738-23e0-44ec-a1ac-faa05a7bcce9 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:56.869269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 666/1356] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:20:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:56.874618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0ed86-5db1-43e3-8ffa-eadf66c903b9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:56.875373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0ed86-5db1-43e3-8ffa-eadf66c903b9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:56.875849 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0ed86-5db1-43e3-8ffa-eadf66c903b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:56.957376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-66a0ed86-5db1-43e3-8ffa-eadf66c903b9 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:56.959410 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-66a0ed86-5db1-43e3-8ffa-eadf66c903b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:57.063274 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 691/1357] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:20:57.365747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebae4d-03be-4dd8-a18d-289f1761405d None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:57.366031 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebae4d-03be-4dd8-a18d-289f1761405d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:57.366307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebae4d-03be-4dd8-a18d-289f1761405d None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:57.367657 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 667/1358] 10.222.0.27 () {54 vars in 962 bytes} [Mon Sep 12 23:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:20:57.375641 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d507bdd-ef06-461b-8457-4c5feaf66489 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:57.376113 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d507bdd-ef06-461b-8457-4c5feaf66489 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:57.376398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d507bdd-ef06-461b-8457-4c5feaf66489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:57.401017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:57.466795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9d507bdd-ef06-461b-8457-4c5feaf66489 None None] MFA Rules not processed for user `f5fa28ffdafd435cb2d93bd453add3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:20:57.468021 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9d507bdd-ef06-461b-8457-4c5feaf66489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:57.592284 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 692/1359] 10.222.0.27 () {58 vars in 1059 bytes} [Mon Sep 12 23:20:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:20:57.601742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebae4d-03be-4dd8-a18d-289f1761405d None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:20:57.604358 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-aaebae4d-03be-4dd8-a18d-289f1761405d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:57.761697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebae4d-03be-4dd8-a18d-289f1761405d None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:20:57.837618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:20:57.880935 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731b386-a8dd-4f24-89a9-0fd32fbda07c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:20:57.881789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731b386-a8dd-4f24-89a9-0fd32fbda07c service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:20:57.882069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731b386-a8dd-4f24-89a9-0fd32fbda07c service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:20:57.882337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731b386-a8dd-4f24-89a9-0fd32fbda07c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:20:57.883949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7731b386-a8dd-4f24-89a9-0fd32fbda07c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:20:57.989251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731b386-a8dd-4f24-89a9-0fd32fbda07c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:20:57.991141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731b386-a8dd-4f24-89a9-0fd32fbda07c service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:20:58.011810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 668/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:00.383243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224a46a-9b0d-42c9-8de0-f8e5d7504996 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:00.383793 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224a46a-9b0d-42c9-8de0-f8e5d7504996 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:00.383911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224a46a-9b0d-42c9-8de0-f8e5d7504996 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:00.385344 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 693/1361] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:21:00.390441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc342b66-7286-4e1d-a0e6-13af61e71140 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:00.390924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc342b66-7286-4e1d-a0e6-13af61e71140 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:00.391039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc342b66-7286-4e1d-a0e6-13af61e71140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:00.481706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-cc342b66-7286-4e1d-a0e6-13af61e71140 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:00.482893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-cc342b66-7286-4e1d-a0e6-13af61e71140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:00.522539 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:00.586950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 669/1362] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:00.875659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224a46a-9b0d-42c9-8de0-f8e5d7504996 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:00.877198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7224a46a-9b0d-42c9-8de0-f8e5d7504996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:00.957861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:01.075773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224a46a-9b0d-42c9-8de0-f8e5d7504996 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:01.203875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606e9c0-c129-43f4-82b1-da7167289866 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:01.204491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606e9c0-c129-43f4-82b1-da7167289866 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:01.204771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606e9c0-c129-43f4-82b1-da7167289866 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:01.205045 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606e9c0-c129-43f4-82b1-da7167289866 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:01.206636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9606e9c0-c129-43f4-82b1-da7167289866 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:01.231827 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:01.300897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606e9c0-c129-43f4-82b1-da7167289866 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:01.302507 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606e9c0-c129-43f4-82b1-da7167289866 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:01.321495 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 694/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:03.787210 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcd9df-9ee0-4555-883f-025ee1099e67 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:03.787796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcd9df-9ee0-4555-883f-025ee1099e67 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:03.787796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcd9df-9ee0-4555-883f-025ee1099e67 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:03.789286 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 670/1364] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:21:03.794359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79173219-a6ae-4629-93ed-000c6a1d8004 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:03.794763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79173219-a6ae-4629-93ed-000c6a1d8004 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:03.794916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79173219-a6ae-4629-93ed-000c6a1d8004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:03.862820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-79173219-a6ae-4629-93ed-000c6a1d8004 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:03.865190 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-79173219-a6ae-4629-93ed-000c6a1d8004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:03.966225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:03.979717 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 695/1365] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:04.260027 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e665506-41c1-4a25-b0fd-0400abf474d8 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:04.260313 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e665506-41c1-4a25-b0fd-0400abf474d8 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:04.260776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e665506-41c1-4a25-b0fd-0400abf474d8 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:04.262398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 671/1366] 10.222.0.27 () {54 vars in 962 bytes} [Mon Sep 12 23:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:04.268542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b58bd2-8860-4b01-9c92-e97f848d444b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:04.269197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b58bd2-8860-4b01-9c92-e97f848d444b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:04.269748 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b58bd2-8860-4b01-9c92-e97f848d444b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:04.351266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a4b58bd2-8860-4b01-9c92-e97f848d444b None None] MFA Rules not processed for user `f5fa28ffdafd435cb2d93bd453add3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:04.352940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a4b58bd2-8860-4b01-9c92-e97f848d444b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:04.477118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 696/1367] 10.222.0.27 () {58 vars in 1059 bytes} [Mon Sep 12 23:21:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:04.485499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e665506-41c1-4a25-b0fd-0400abf474d8 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:04.487255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3e665506-41c1-4a25-b0fd-0400abf474d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:04.643178 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e665506-41c1-4a25-b0fd-0400abf474d8 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:04.667933 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:04.767433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dde290-5efc-4759-994b-dfa79e3399a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:04.768075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dde290-5efc-4759-994b-dfa79e3399a8 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:04.768357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dde290-5efc-4759-994b-dfa79e3399a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:04.768636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dde290-5efc-4759-994b-dfa79e3399a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:04.770782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-56dde290-5efc-4759-994b-dfa79e3399a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:04.866910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dde290-5efc-4759-994b-dfa79e3399a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:04.870505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dde290-5efc-4759-994b-dfa79e3399a8 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:04.891563 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 672/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:05.015781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161f56c-3365-4dbe-92f4-531c582ac056 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:05.016346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161f56c-3365-4dbe-92f4-531c582ac056 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:05.016346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161f56c-3365-4dbe-92f4-531c582ac056 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:05.020096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 697/1369] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:05.024184 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec4091-a467-46ca-afcb-9ff574555ef8 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:05.024725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec4091-a467-46ca-afcb-9ff574555ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:05.025019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec4091-a467-46ca-afcb-9ff574555ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:05.061590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:05.113848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b7ec4091-a467-46ca-afcb-9ff574555ef8 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:05.115109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b7ec4091-a467-46ca-afcb-9ff574555ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:05.239086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 673/1370] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:05.575373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd21a5-d84b-4bd4-8d52-057f939031c9 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:05.575640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd21a5-d84b-4bd4-8d52-057f939031c9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:05.575889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd21a5-d84b-4bd4-8d52-057f939031c9 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:05.577415 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 698/1371] 10.222.0.27 () {54 vars in 964 bytes} [Mon Sep 12 23:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:05.582280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b25802-5e97-461a-8b29-87511b3c81b9 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:05.582719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b25802-5e97-461a-8b29-87511b3c81b9 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:05.583108 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b25802-5e97-461a-8b29-87511b3c81b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:05.669093 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-74b25802-5e97-461a-8b29-87511b3c81b9 None None] MFA Rules not processed for user `c5b8828077594349a93c7a842c0f15da`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:05.670420 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-74b25802-5e97-461a-8b29-87511b3c81b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:05.719667 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:05.752233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 674/1372] 10.222.0.27 () {58 vars in 1061 bytes} [Mon Sep 12 23:21:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:21:05.762085 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd21a5-d84b-4bd4-8d52-057f939031c9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:05.763666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2bcd21a5-d84b-4bd4-8d52-057f939031c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:05.817071 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:05.872484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd21a5-d84b-4bd4-8d52-057f939031c9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:05.959412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a6651-8059-4891-9b77-95acfd76dc71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:05.960045 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a6651-8059-4891-9b77-95acfd76dc71 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:05.960321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a6651-8059-4891-9b77-95acfd76dc71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:05.960589 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a6651-8059-4891-9b77-95acfd76dc71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:05.962162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-749a6651-8059-4891-9b77-95acfd76dc71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:06.077132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:06.096397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a6651-8059-4891-9b77-95acfd76dc71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:06.098438 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a6651-8059-4891-9b77-95acfd76dc71 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:06.125153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 699/1373] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:07.195269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e8b6f-aaed-4d37-9202-a07b38207d1e None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:07.195911 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e8b6f-aaed-4d37-9202-a07b38207d1e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:07.195911 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e8b6f-aaed-4d37-9202-a07b38207d1e None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:07.197194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 675/1374] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:07.202370 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36387a-5240-4eea-b49b-e80f3974b999 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:07.202778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36387a-5240-4eea-b49b-e80f3974b999 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:07.202921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36387a-5240-4eea-b49b-e80f3974b999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:07.267697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dd36387a-5240-4eea-b49b-e80f3974b999 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:07.268925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dd36387a-5240-4eea-b49b-e80f3974b999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:07.366934 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 700/1375] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:07.668563 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e8b6f-aaed-4d37-9202-a07b38207d1e None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:07.670128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2b7e8b6f-aaed-4d37-9202-a07b38207d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:07.823739 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e8b6f-aaed-4d37-9202-a07b38207d1e None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:07.877276 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:07.945546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b5b84-7c3a-4eaf-b1b2-13d2838aee47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:07.946149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b5b84-7c3a-4eaf-b1b2-13d2838aee47 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:07.946425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b5b84-7c3a-4eaf-b1b2-13d2838aee47 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:07.946694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b5b84-7c3a-4eaf-b1b2-13d2838aee47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:07.948307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-880b5b84-7c3a-4eaf-b1b2-13d2838aee47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:08.042964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b5b84-7c3a-4eaf-b1b2-13d2838aee47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:08.044615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b5b84-7c3a-4eaf-b1b2-13d2838aee47 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:08.064718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 676/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:10.272839 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60f027-5af0-4e18-adde-c55de6c16af0 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:10.273369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60f027-5af0-4e18-adde-c55de6c16af0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:10.273369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60f027-5af0-4e18-adde-c55de6c16af0 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:10.275042 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 701/1377] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:10.280362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0539b0-996a-4fb8-8928-5ef7f7ee4d05 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:10.280786 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0539b0-996a-4fb8-8928-5ef7f7ee4d05 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:10.280912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0539b0-996a-4fb8-8928-5ef7f7ee4d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:10.347854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-7c0539b0-996a-4fb8-8928-5ef7f7ee4d05 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:10.349053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7c0539b0-996a-4fb8-8928-5ef7f7ee4d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:10.355744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:10.452641 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 677/1378] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:10.738336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60f027-5af0-4e18-adde-c55de6c16af0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:10.740035 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dc60f027-5af0-4e18-adde-c55de6c16af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:10.805377 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:10.895345 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60f027-5af0-4e18-adde-c55de6c16af0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:11.021879 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d76400-b088-4832-ab2d-87103821984d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:11.022396 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d76400-b088-4832-ab2d-87103821984d service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:11.022676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d76400-b088-4832-ab2d-87103821984d service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:11.022939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d76400-b088-4832-ab2d-87103821984d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:11.024785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-37d76400-b088-4832-ab2d-87103821984d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:11.044759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:11.118318 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d76400-b088-4832-ab2d-87103821984d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:11.120098 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d76400-b088-4832-ab2d-87103821984d service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:11.139346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 702/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:13.329973 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c100d3d-180b-400e-b187-d6b15c028e52 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:13.330536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c100d3d-180b-400e-b187-d6b15c028e52 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:13.330536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c100d3d-180b-400e-b187-d6b15c028e52 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:13.332746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 678/1380] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:21:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:13.338242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e779957-bb98-4045-bc64-d3d796114234 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:13.339003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e779957-bb98-4045-bc64-d3d796114234 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:13.339442 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e779957-bb98-4045-bc64-d3d796114234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:13.409953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8e779957-bb98-4045-bc64-d3d796114234 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:13.411226 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8e779957-bb98-4045-bc64-d3d796114234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:13.546172 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:13.555608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 703/1381] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:13.846872 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c100d3d-180b-400e-b187-d6b15c028e52 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:13.849052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-0c100d3d-180b-400e-b187-d6b15c028e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:13.994832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:14.006130 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c100d3d-180b-400e-b187-d6b15c028e52 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:14.137241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ddd88-44aa-4e16-8066-9486b727d515 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:14.137770 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ddd88-44aa-4e16-8066-9486b727d515 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:14.138021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ddd88-44aa-4e16-8066-9486b727d515 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:14.138288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ddd88-44aa-4e16-8066-9486b727d515 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:14.139934 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9a7ddd88-44aa-4e16-8066-9486b727d515 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:14.236459 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ddd88-44aa-4e16-8066-9486b727d515 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:14.238792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ddd88-44aa-4e16-8066-9486b727d515 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:14.249346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:14.257350 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 679/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:16.962925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70800715-8367-40db-b2a4-8feb4ca0a709 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:16.963650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70800715-8367-40db-b2a4-8feb4ca0a709 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:16.963650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70800715-8367-40db-b2a4-8feb4ca0a709 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:16.966146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 704/1383] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:21:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:21:16.971598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333332b-8922-4a8e-b409-19d37b8dd072 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:16.972042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333332b-8922-4a8e-b409-19d37b8dd072 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:16.972734 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333332b-8922-4a8e-b409-19d37b8dd072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:17.040923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-2333332b-8922-4a8e-b409-19d37b8dd072 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:17.042172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2333332b-8922-4a8e-b409-19d37b8dd072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:17.137793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 680/1384] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:17.426887 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70800715-8367-40db-b2a4-8feb4ca0a709 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:17.428418 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-70800715-8367-40db-b2a4-8feb4ca0a709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:17.578050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70800715-8367-40db-b2a4-8feb4ca0a709 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:17.648706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:17.698781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7e175-2b08-433f-ab9d-cd182dcc83c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:17.699308 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e175-2b08-433f-ab9d-cd182dcc83c3 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:17.699607 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e175-2b08-433f-ab9d-cd182dcc83c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:17.699881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e175-2b08-433f-ab9d-cd182dcc83c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:17.701467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b0b7e175-2b08-433f-ab9d-cd182dcc83c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:17.798280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7e175-2b08-433f-ab9d-cd182dcc83c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:17.799985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7e175-2b08-433f-ab9d-cd182dcc83c3 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:17.818437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 705/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:19.996781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf1154-d10a-40fe-9ffc-6e771857c8b3 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:19.997479 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf1154-d10a-40fe-9ffc-6e771857c8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:19.997479 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf1154-d10a-40fe-9ffc-6e771857c8b3 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:19.998908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 681/1386] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:20.004818 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2ced6-4b9c-4724-b1ba-8bd09b359bbf None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:20.005311 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2ced6-4b9c-4724-b1ba-8bd09b359bbf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:20.005476 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2ced6-4b9c-4724-b1ba-8bd09b359bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:20.073951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e6e2ced6-4b9c-4724-b1ba-8bd09b359bbf None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:20.075820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e6e2ced6-4b9c-4724-b1ba-8bd09b359bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:20.109760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:20.174326 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 706/1387] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:21:20.465314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf1154-d10a-40fe-9ffc-6e771857c8b3 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:20.466884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-0edf1154-d10a-40fe-9ffc-6e771857c8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:20.544716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:20.622031 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf1154-d10a-40fe-9ffc-6e771857c8b3 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:20.746053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50b9fd-cafe-45dd-99cc-102be5d59e2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:20.746665 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50b9fd-cafe-45dd-99cc-102be5d59e2c service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:20.746946 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50b9fd-cafe-45dd-99cc-102be5d59e2c service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:20.747222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50b9fd-cafe-45dd-99cc-102be5d59e2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:20.748959 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-fc50b9fd-cafe-45dd-99cc-102be5d59e2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:20.793082 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:20.852113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50b9fd-cafe-45dd-99cc-102be5d59e2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:20.853945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50b9fd-cafe-45dd-99cc-102be5d59e2c service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:20.873875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 682/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:23.041886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0892283-4dff-4395-a25f-3eb8af6893b3 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:23.042318 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0892283-4dff-4395-a25f-3eb8af6893b3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:23.042547 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0892283-4dff-4395-a25f-3eb8af6893b3 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:23.045306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 707/1389] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:23.049667 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344e122-4c12-4560-aa85-59a49782b119 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:23.050267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344e122-4c12-4560-aa85-59a49782b119 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:23.050267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344e122-4c12-4560-aa85-59a49782b119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:23.114805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-7344e122-4c12-4560-aa85-59a49782b119 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:23.116015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7344e122-4c12-4560-aa85-59a49782b119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:23.211947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 683/1390] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:21:23.510692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0892283-4dff-4395-a25f-3eb8af6893b3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:23.512319 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d0892283-4dff-4395-a25f-3eb8af6893b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:23.663195 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0892283-4dff-4395-a25f-3eb8af6893b3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:23.680167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:23.789717 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ddd87-5162-4e60-af3d-91717337ae53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:23.790329 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ddd87-5162-4e60-af3d-91717337ae53 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:23.790620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ddd87-5162-4e60-af3d-91717337ae53 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:23.790909 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ddd87-5162-4e60-af3d-91717337ae53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:23.792647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0e6ddd87-5162-4e60-af3d-91717337ae53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:23.897762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ddd87-5162-4e60-af3d-91717337ae53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:23.899589 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ddd87-5162-4e60-af3d-91717337ae53 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:23.923024 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 708/1391] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:24.081498 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b77080-7089-4bd5-bd25-ca131ef9820a None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:24.082072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b77080-7089-4bd5-bd25-ca131ef9820a None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:24.082072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b77080-7089-4bd5-bd25-ca131ef9820a None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:24.083376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 684/1392] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:24.089117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d77655-069d-4d50-9d6e-90d719d73996 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:24.089660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d77655-069d-4d50-9d6e-90d719d73996 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:24.089660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d77655-069d-4d50-9d6e-90d719d73996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:24.119037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:24.165743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b0d77655-069d-4d50-9d6e-90d719d73996 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:24.167245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b0d77655-069d-4d50-9d6e-90d719d73996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:24.263516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 709/1393] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:24.562817 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ae531-5a01-4143-bb1a-cf84e61c56fe None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:24.563196 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ae531-5a01-4143-bb1a-cf84e61c56fe None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:24.563540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ae531-5a01-4143-bb1a-cf84e61c56fe None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:24.565008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 685/1394] 10.222.0.27 () {54 vars in 966 bytes} [Mon Sep 12 23:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:24.570822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa5dc-1969-4b18-b5ea-d5b364ae03da None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:24.571182 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa5dc-1969-4b18-b5ea-d5b364ae03da None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:24.571551 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa5dc-1969-4b18-b5ea-d5b364ae03da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:24.654287 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0dafa5dc-1969-4b18-b5ea-d5b364ae03da None None] MFA Rules not processed for user `d61407406ef242a78030f79532fe2247`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:24.655516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0dafa5dc-1969-4b18-b5ea-d5b364ae03da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:24.697911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:24.735245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 710/1395] 10.222.0.27 () {58 vars in 1063 bytes} [Mon Sep 12 23:21:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:24.745118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ae531-5a01-4143-bb1a-cf84e61c56fe None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:24.746708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-626ae531-5a01-4143-bb1a-cf84e61c56fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:24.760408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:24.857576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ae531-5a01-4143-bb1a-cf84e61c56fe None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:24.934440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfa2fa-c8ee-4b74-9ad0-05db7ca5d68f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61407406ef242a78030f79532fe2247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:24.935068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfa2fa-c8ee-4b74-9ad0-05db7ca5d68f service glance] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:24.935326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfa2fa-c8ee-4b74-9ad0-05db7ca5d68f service glance] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:24.935618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfa2fa-c8ee-4b74-9ad0-05db7ca5d68f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:24.937208 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-fbcfa2fa-c8ee-4b74-9ad0-05db7ca5d68f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:25.008249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:25.053914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfa2fa-c8ee-4b74-9ad0-05db7ca5d68f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:25.055587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfa2fa-c8ee-4b74-9ad0-05db7ca5d68f service glance] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:25.073791 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 686/1396] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:25.892925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf61c95-8137-4a44-af5b-0efe7bc80789 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:25.893420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf61c95-8137-4a44-af5b-0efe7bc80789 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:25.893505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf61c95-8137-4a44-af5b-0efe7bc80789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:25.967814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-cbf61c95-8137-4a44-af5b-0efe7bc80789 None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:25.969020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cbf61c95-8137-4a44-af5b-0efe7bc80789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:26.060356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 711/1397] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:21:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:26.102940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f54654-4785-4cbd-b7c2-38a5769d352e None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:26.103431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f54654-4785-4cbd-b7c2-38a5769d352e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:26.103529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f54654-4785-4cbd-b7c2-38a5769d352e None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:26.104869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 687/1398] 10.222.0.27 () {54 vars in 943 bytes} [Mon Sep 12 23:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:26.111598 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad086e3-1351-41fe-a0c1-bf9daee1c3a3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:26.111884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad086e3-1351-41fe-a0c1-bf9daee1c3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:26.112163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad086e3-1351-41fe-a0c1-bf9daee1c3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:26.120236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25682e20-1322-40e4-aa43-038c63d1f3a2 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:26.120530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25682e20-1322-40e4-aa43-038c63d1f3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:26.120809 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25682e20-1322-40e4-aa43-038c63d1f3a2 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:26.122163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 688/1399] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:21:26.126954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7108de5-3e32-4a87-a517-0e1f98b81719 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:26.127307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7108de5-3e32-4a87-a517-0e1f98b81719 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:26.127685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7108de5-3e32-4a87-a517-0e1f98b81719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:26.182459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:26.190493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7ad086e3-1351-41fe-a0c1-bf9daee1c3a3 None None] MFA Rules not processed for user `c1f6bb09fd7f4905a1693ef7cb6f088e`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:26.191746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7ad086e3-1351-41fe-a0c1-bf9daee1c3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:26.194328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b7108de5-3e32-4a87-a517-0e1f98b81719 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:26.195511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b7108de5-3e32-4a87-a517-0e1f98b81719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:26.293701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 689/1400] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:21:26.299935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 712/1401] 10.222.0.27 () {58 vars in 1040 bytes} [Mon Sep 12 23:21:26 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:26.310006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7108de5-3e32-4a87-a517-0e1f98b81719 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:26.311178 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b7108de5-3e32-4a87-a517-0e1f98b81719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:26.337668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:26.413835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7108de5-3e32-4a87-a517-0e1f98b81719 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:26.515794 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e35e9e-981c-4bb1-a0fc-547f2d54e00c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:26.516335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e35e9e-981c-4bb1-a0fc-547f2d54e00c service swift] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:26.516621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e35e9e-981c-4bb1-a0fc-547f2d54e00c service swift] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:26.516900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e35e9e-981c-4bb1-a0fc-547f2d54e00c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:26.518542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b7e35e9e-981c-4bb1-a0fc-547f2d54e00c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:26.558771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:26.588842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad086e3-1351-41fe-a0c1-bf9daee1c3a3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:26.590084 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7ad086e3-1351-41fe-a0c1-bf9daee1c3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:26.603871 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e35e9e-981c-4bb1-a0fc-547f2d54e00c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:26.605772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e35e9e-981c-4bb1-a0fc-547f2d54e00c service swift] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:26.614288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 690/1402] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:21:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:21:26.694802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad086e3-1351-41fe-a0c1-bf9daee1c3a3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:26.717326 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:26.823556 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99e8a0-ca20-448a-a041-48ccc5a0f1f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:26.824109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99e8a0-ca20-448a-a041-48ccc5a0f1f8 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:26.824386 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99e8a0-ca20-448a-a041-48ccc5a0f1f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:26.824655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99e8a0-ca20-448a-a041-48ccc5a0f1f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:26.826274 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9e99e8a0-ca20-448a-a041-48ccc5a0f1f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:26.926631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99e8a0-ca20-448a-a041-48ccc5a0f1f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:26.928299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99e8a0-ca20-448a-a041-48ccc5a0f1f8 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:26.947691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 713/1403] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:21:29.183505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc186b2f-9e06-467f-be5c-be9b776f2792 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:29.184130 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc186b2f-9e06-467f-be5c-be9b776f2792 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:29.184130 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc186b2f-9e06-467f-be5c-be9b776f2792 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:29.185564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 691/1404] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:29.190749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e0c31-a6df-45a0-9b7b-f5388ca9f3af None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:29.191159 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e0c31-a6df-45a0-9b7b-f5388ca9f3af None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:29.191870 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e0c31-a6df-45a0-9b7b-f5388ca9f3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:29.223660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:29.260107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1f6e0c31-a6df-45a0-9b7b-f5388ca9f3af None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:29.261468 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1f6e0c31-a6df-45a0-9b7b-f5388ca9f3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:29.365597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 714/1405] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:29.657397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc186b2f-9e06-467f-be5c-be9b776f2792 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:29.658906 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-cc186b2f-9e06-467f-be5c-be9b776f2792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:29.826164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc186b2f-9e06-467f-be5c-be9b776f2792 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:29.872796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:30.052058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381ddcb-1658-4c96-9f9b-e931f3dc1789 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:30.052781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381ddcb-1658-4c96-9f9b-e931f3dc1789 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:30.053121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381ddcb-1658-4c96-9f9b-e931f3dc1789 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:30.053876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381ddcb-1658-4c96-9f9b-e931f3dc1789 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:30.055552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-0381ddcb-1658-4c96-9f9b-e931f3dc1789 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:30.161663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381ddcb-1658-4c96-9f9b-e931f3dc1789 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:30.163272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381ddcb-1658-4c96-9f9b-e931f3dc1789 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:30.183157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 692/1406] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:32.400580 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab606f1-2859-4bf1-b3ba-f53c7c2feb02 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:32.401117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab606f1-2859-4bf1-b3ba-f53c7c2feb02 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:32.401117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab606f1-2859-4bf1-b3ba-f53c7c2feb02 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:32.402379 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 715/1407] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:32.407797 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cbd03-f470-4ffd-8086-cec1d5ff3fdc None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:32.408541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cbd03-f470-4ffd-8086-cec1d5ff3fdc None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:32.409049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cbd03-f470-4ffd-8086-cec1d5ff3fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:32.462984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:32.484404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-2b7cbd03-f470-4ffd-8086-cec1d5ff3fdc None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:32.485636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2b7cbd03-f470-4ffd-8086-cec1d5ff3fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:32.591314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 693/1408] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:32.879997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab606f1-2859-4bf1-b3ba-f53c7c2feb02 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:32.881753 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3ab606f1-2859-4bf1-b3ba-f53c7c2feb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:33.004609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:33.065847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab606f1-2859-4bf1-b3ba-f53c7c2feb02 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:33.192578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4c3cf-5d06-4430-a5e6-30c8c60f0769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:33.193292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4c3cf-5d06-4430-a5e6-30c8c60f0769 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:33.193567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4c3cf-5d06-4430-a5e6-30c8c60f0769 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:33.193838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4c3cf-5d06-4430-a5e6-30c8c60f0769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:33.195454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6eb4c3cf-5d06-4430-a5e6-30c8c60f0769 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:33.283474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:33.308028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4c3cf-5d06-4430-a5e6-30c8c60f0769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:33.309665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4c3cf-5d06-4430-a5e6-30c8c60f0769 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:33.333691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 716/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:42.589625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=47 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:43.330131 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=16 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:49.313211 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e7dd5-0a07-407e-98eb-7fe02b33cfea None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:49.313775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e7dd5-0a07-407e-98eb-7fe02b33cfea None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:49.313775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e7dd5-0a07-407e-98eb-7fe02b33cfea None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:49.315087 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 694/1410] 10.222.0.27 () {54 vars in 918 bytes} [Mon Sep 12 23:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:49.320778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170585e-c5e2-4968-af5a-b3dfa3c95109 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:49.321223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170585e-c5e2-4968-af5a-b3dfa3c95109 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:49.321337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170585e-c5e2-4968-af5a-b3dfa3c95109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:49.396254 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7170585e-c5e2-4968-af5a-b3dfa3c95109 None None] MFA Rules not processed for user `2b3b0a0317b14e29a32eb6ba6b89c2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:49.397458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7170585e-c5e2-4968-af5a-b3dfa3c95109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:49.506709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 717/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Mon Sep 12 23:21:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:49.521421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f09983-ef2c-45cd-9f6b-8725270930c0 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:49.521762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f09983-ef2c-45cd-9f6b-8725270930c0 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:49.522039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f09983-ef2c-45cd-9f6b-8725270930c0 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:49.523376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 695/1412] 10.222.0.27 () {54 vars in 918 bytes} [Mon Sep 12 23:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:49.529306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02000b3-aac8-442b-9620-b28d0bfc3ea9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:49.529592 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02000b3-aac8-442b-9620-b28d0bfc3ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:49.529865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02000b3-aac8-442b-9620-b28d0bfc3ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:49.602793 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:49.606401 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d02000b3-aac8-442b-9620-b28d0bfc3ea9 None None] MFA Rules not processed for user `f5fa28ffdafd435cb2d93bd453add3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:49.608182 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d02000b3-aac8-442b-9620-b28d0bfc3ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:49.722621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 718/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Mon Sep 12 23:21:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:52.277658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37617409-b08c-4321-95ae-6057e76ba6b1 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:52.278186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37617409-b08c-4321-95ae-6057e76ba6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:52.278186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37617409-b08c-4321-95ae-6057e76ba6b1 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:52.279499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 696/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:52.284678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7f12c-7262-4490-bc34-e61f99b0de77 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:52.285579 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7f12c-7262-4490-bc34-e61f99b0de77 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:52.285792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7f12c-7262-4490-bc34-e61f99b0de77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:52.350487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-36d7f12c-7262-4490-bc34-e61f99b0de77 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:52.351724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-36d7f12c-7262-4490-bc34-e61f99b0de77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:52.406326 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:52.447309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 719/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:52.735938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37617409-b08c-4321-95ae-6057e76ba6b1 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:52.737453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-37617409-b08c-4321-95ae-6057e76ba6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:52.881731 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37617409-b08c-4321-95ae-6057e76ba6b1 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:52.954921 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:52.994016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595811b-f7e7-4203-b0cf-e2ba908e88d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:52.994546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595811b-f7e7-4203-b0cf-e2ba908e88d3 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:52.994822 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595811b-f7e7-4203-b0cf-e2ba908e88d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:52.995096 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595811b-f7e7-4203-b0cf-e2ba908e88d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:52.996699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5595811b-f7e7-4203-b0cf-e2ba908e88d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:53.086758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595811b-f7e7-4203-b0cf-e2ba908e88d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:53.088410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595811b-f7e7-4203-b0cf-e2ba908e88d3 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:53.106873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 697/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:54.254923 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235fac7-e23f-435e-b52d-b386c3bdf598 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:54.255450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235fac7-e23f-435e-b52d-b386c3bdf598 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:54.255450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235fac7-e23f-435e-b52d-b386c3bdf598 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:54.256742 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 720/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:54.262957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc5fb1-6f25-4d40-923a-b0dc05379750 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:54.263388 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc5fb1-6f25-4d40-923a-b0dc05379750 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:54.263521 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc5fb1-6f25-4d40-923a-b0dc05379750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:54.337912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b6cc5fb1-6f25-4d40-923a-b0dc05379750 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:54.339114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b6cc5fb1-6f25-4d40-923a-b0dc05379750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:54.366401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:54.436652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 698/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:54.589596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235fac7-e23f-435e-b52d-b386c3bdf598 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:54.591045 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7235fac7-e23f-435e-b52d-b386c3bdf598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:54.734761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235fac7-e23f-435e-b52d-b386c3bdf598 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:54.782618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:54.856383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1de363-f19a-4378-96ed-24106914823f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:54.856991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1de363-f19a-4378-96ed-24106914823f service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:54.857267 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1de363-f19a-4378-96ed-24106914823f service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:54.857535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1de363-f19a-4378-96ed-24106914823f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:54.859115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-be1de363-f19a-4378-96ed-24106914823f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:54.949594 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1de363-f19a-4378-96ed-24106914823f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:54.951211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1de363-f19a-4378-96ed-24106914823f service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:54.968651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 721/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:56.094604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb18814-7044-4e92-9e9d-7c7f61687851 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:56.095167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb18814-7044-4e92-9e9d-7c7f61687851 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:56.095167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb18814-7044-4e92-9e9d-7c7f61687851 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:56.096537 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 699/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:56.103360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe7cb-8e23-4a4b-b526-eee46710f858 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:56.103862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe7cb-8e23-4a4b-b526-eee46710f858 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:56.103973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe7cb-8e23-4a4b-b526-eee46710f858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:56.156651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:56.178989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-eedfe7cb-8e23-4a4b-b526-eee46710f858 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:56.180158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eedfe7cb-8e23-4a4b-b526-eee46710f858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:56.281118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 722/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:21:56.433021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb18814-7044-4e92-9e9d-7c7f61687851 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:56.434443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4fb18814-7044-4e92-9e9d-7c7f61687851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:56.580670 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb18814-7044-4e92-9e9d-7c7f61687851 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:56.593016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:56.696462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8ae95-256e-408e-921b-d07b0997f0d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:56.696990 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8ae95-256e-408e-921b-d07b0997f0d8 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:56.697270 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8ae95-256e-408e-921b-d07b0997f0d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:56.697543 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8ae95-256e-408e-921b-d07b0997f0d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:56.699126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4bc8ae95-256e-408e-921b-d07b0997f0d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:56.786711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8ae95-256e-408e-921b-d07b0997f0d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:56.788326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8ae95-256e-408e-921b-d07b0997f0d8 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:56.805534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 700/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:57.913453 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cbb78-729d-474b-95bb-6886b40beb08 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:57.914071 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cbb78-729d-474b-95bb-6886b40beb08 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:57.914071 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cbb78-729d-474b-95bb-6886b40beb08 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:57.915887 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 723/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:57.922032 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d881d-909c-47c7-b40b-222b693b013f None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:57.922492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d881d-909c-47c7-b40b-222b693b013f None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:57.922598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d881d-909c-47c7-b40b-222b693b013f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:57.943183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:57.999260 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-527d881d-909c-47c7-b40b-222b693b013f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:58.000485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-527d881d-909c-47c7-b40b-222b693b013f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:58.094108 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 701/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:21:58.241765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cbb78-729d-474b-95bb-6886b40beb08 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:21:58.243361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ea0cbb78-729d-474b-95bb-6886b40beb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:58.352091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:58.387065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cbb78-729d-474b-95bb-6886b40beb08 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:21:58.500439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a53a0b-a4f1-4614-963f-263c1c31b676 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:21:58.500969 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a53a0b-a4f1-4614-963f-263c1c31b676 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:58.501253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a53a0b-a4f1-4614-963f-263c1c31b676 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:58.501525 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a53a0b-a4f1-4614-963f-263c1c31b676 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:58.503107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-76a53a0b-a4f1-4614-963f-263c1c31b676 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:58.579316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:21:58.593755 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a53a0b-a4f1-4614-963f-263c1c31b676 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:21:58.595364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a53a0b-a4f1-4614-963f-263c1c31b676 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:21:58.614434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 724/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:21:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:21:59.719288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef610a2-c815-4af2-80e5-8502d3ec6e85 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:59.719812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef610a2-c815-4af2-80e5-8502d3ec6e85 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:59.719923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef610a2-c815-4af2-80e5-8502d3ec6e85 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:59.721230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 702/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:21:59.729028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccdd24-10f8-4c01-a55c-50caedb8f8d6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:21:59.729483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccdd24-10f8-4c01-a55c-50caedb8f8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:21:59.729585 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccdd24-10f8-4c01-a55c-50caedb8f8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:21:59.803283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-65ccdd24-10f8-4c01-a55c-50caedb8f8d6 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:21:59.804517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-65ccdd24-10f8-4c01-a55c-50caedb8f8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:21:59.898666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 725/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:00.048527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef610a2-c815-4af2-80e5-8502d3ec6e85 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:00.050054 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7ef610a2-c815-4af2-80e5-8502d3ec6e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:00.132773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:00.193289 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef610a2-c815-4af2-80e5-8502d3ec6e85 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:00.302722 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac285a7f-c90d-4c0f-81dd-b7f4e3275e16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:00.303327 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac285a7f-c90d-4c0f-81dd-b7f4e3275e16 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:00.303629 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac285a7f-c90d-4c0f-81dd-b7f4e3275e16 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:00.303905 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac285a7f-c90d-4c0f-81dd-b7f4e3275e16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:00.305492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ac285a7f-c90d-4c0f-81dd-b7f4e3275e16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:00.352900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:00.394352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac285a7f-c90d-4c0f-81dd-b7f4e3275e16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:00.395992 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac285a7f-c90d-4c0f-81dd-b7f4e3275e16 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:00.413526 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 703/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:01.466815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e346c-99c4-4cf8-9899-32a835f96e94 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:01.467370 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e346c-99c4-4cf8-9899-32a835f96e94 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:01.467370 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e346c-99c4-4cf8-9899-32a835f96e94 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:01.468729 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 726/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:01.474937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd4ce3-9e78-46bb-aef7-7547098b29be None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:01.475386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd4ce3-9e78-46bb-aef7-7547098b29be None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:01.475558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd4ce3-9e78-46bb-aef7-7547098b29be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:01.551629 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-bcbd4ce3-9e78-46bb-aef7-7547098b29be None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:01.552902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bcbd4ce3-9e78-46bb-aef7-7547098b29be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:01.652671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 704/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:01.799298 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e346c-99c4-4cf8-9899-32a835f96e94 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:01.801368 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-740e346c-99c4-4cf8-9899-32a835f96e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:01.849590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:01.946853 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e346c-99c4-4cf8-9899-32a835f96e94 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:02.060473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ee491-9dab-45c8-89c5-2f2941b2e0be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:02.061078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ee491-9dab-45c8-89c5-2f2941b2e0be service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:02.061356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ee491-9dab-45c8-89c5-2f2941b2e0be service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:02.061629 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ee491-9dab-45c8-89c5-2f2941b2e0be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:02.063237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cb2ee491-9dab-45c8-89c5-2f2941b2e0be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:02.070787 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:02.152861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ee491-9dab-45c8-89c5-2f2941b2e0be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:02.154452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ee491-9dab-45c8-89c5-2f2941b2e0be service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:02.171956 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 727/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:03.221610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95b07e-6cbc-4ef3-98dc-4e17cccfbf36 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:03.222143 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95b07e-6cbc-4ef3-98dc-4e17cccfbf36 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:03.222143 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95b07e-6cbc-4ef3-98dc-4e17cccfbf36 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:03.224094 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 705/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:22:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:03.230231 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9fb80-6edc-497e-9bc4-9c012d6f2cb7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:03.230649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9fb80-6edc-497e-9bc4-9c012d6f2cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:03.230783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9fb80-6edc-497e-9bc4-9c012d6f2cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:03.304552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c1f9fb80-6edc-497e-9bc4-9c012d6f2cb7 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:03.305754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c1f9fb80-6edc-497e-9bc4-9c012d6f2cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:03.371693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:03.400755 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 728/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:03.550625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95b07e-6cbc-4ef3-98dc-4e17cccfbf36 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:03.552177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7f95b07e-6cbc-4ef3-98dc-4e17cccfbf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:03.571131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:03.698976 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95b07e-6cbc-4ef3-98dc-4e17cccfbf36 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:03.788784 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:03.812373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670261f-ae13-4389-b51c-a85bf1f9966b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:03.812910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670261f-ae13-4389-b51c-a85bf1f9966b service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:03.813187 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670261f-ae13-4389-b51c-a85bf1f9966b service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:03.813455 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670261f-ae13-4389-b51c-a85bf1f9966b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:03.815030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3670261f-ae13-4389-b51c-a85bf1f9966b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:03.906918 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670261f-ae13-4389-b51c-a85bf1f9966b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:03.908580 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670261f-ae13-4389-b51c-a85bf1f9966b service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:03.927368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 706/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:04.983700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3841fa-d0c8-4b18-aede-7c231bdca7e0 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:04.984201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3841fa-d0c8-4b18-aede-7c231bdca7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:04.984300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3841fa-d0c8-4b18-aede-7c231bdca7e0 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:04.985588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 729/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:04.991699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecee845-569c-4a85-a6ba-8e3b9a326deb None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:04.992100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecee845-569c-4a85-a6ba-8e3b9a326deb None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:04.992252 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecee845-569c-4a85-a6ba-8e3b9a326deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:05.067737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-4ecee845-569c-4a85-a6ba-8e3b9a326deb None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:05.068964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4ecee845-569c-4a85-a6ba-8e3b9a326deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:05.108936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:05.163275 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 707/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:05.314747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3841fa-d0c8-4b18-aede-7c231bdca7e0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:05.316220 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bf3841fa-d0c8-4b18-aede-7c231bdca7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:05.460815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3841fa-d0c8-4b18-aede-7c231bdca7e0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:05.514655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:05.572518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95446c87-4f43-473f-a113-d936528700d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:05.573054 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95446c87-4f43-473f-a113-d936528700d5 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:05.573330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95446c87-4f43-473f-a113-d936528700d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:05.573599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95446c87-4f43-473f-a113-d936528700d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:05.575184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-95446c87-4f43-473f-a113-d936528700d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:05.665703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95446c87-4f43-473f-a113-d936528700d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:05.667298 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95446c87-4f43-473f-a113-d936528700d5 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:05.685500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 730/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:06.722556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb9a76-59c0-40fb-b8fe-4c21515657af None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:06.723650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb9a76-59c0-40fb-b8fe-4c21515657af None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:06.723650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb9a76-59c0-40fb-b8fe-4c21515657af None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:06.725013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 708/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:22:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:06.730967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8abe0-5a05-4c6f-bee5-db2676211abe None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:06.731383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8abe0-5a05-4c6f-bee5-db2676211abe None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:06.731518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8abe0-5a05-4c6f-bee5-db2676211abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:06.805103 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bfb8abe0-5a05-4c6f-bee5-db2676211abe None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:06.806294 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bfb8abe0-5a05-4c6f-bee5-db2676211abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:06.808312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:06.902923 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 731/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:07.050348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb9a76-59c0-40fb-b8fe-4c21515657af None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:07.051843 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e8bb9a76-59c0-40fb-b8fe-4c21515657af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:07.192978 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb9a76-59c0-40fb-b8fe-4c21515657af None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:07.219233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:07.312669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb034d-bfd9-4551-b8e3-5a98ac2a2563 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:07.313186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb034d-bfd9-4551-b8e3-5a98ac2a2563 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:07.313464 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb034d-bfd9-4551-b8e3-5a98ac2a2563 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:07.313736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb034d-bfd9-4551-b8e3-5a98ac2a2563 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:07.315318 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5bdb034d-bfd9-4551-b8e3-5a98ac2a2563 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:07.407397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb034d-bfd9-4551-b8e3-5a98ac2a2563 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:07.409079 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb034d-bfd9-4551-b8e3-5a98ac2a2563 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:07.426763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 709/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:09.493273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3b566-c636-4f2b-9327-42ec9dd1ddc4 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:09.493832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3b566-c636-4f2b-9327-42ec9dd1ddc4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:09.493832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3b566-c636-4f2b-9327-42ec9dd1ddc4 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:09.495141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 732/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:09.500446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92547a-ffed-4812-9980-5f854dec01bb None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:09.500866 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92547a-ffed-4812-9980-5f854dec01bb None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:09.501023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92547a-ffed-4812-9980-5f854dec01bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:09.543102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:09.568699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-7e92547a-ffed-4812-9980-5f854dec01bb None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:09.569915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7e92547a-ffed-4812-9980-5f854dec01bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:09.668210 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 710/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:09.951913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3b566-c636-4f2b-9327-42ec9dd1ddc4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:09.953459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3df3b566-c636-4f2b-9327-42ec9dd1ddc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:10.056124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3b566-c636-4f2b-9327-42ec9dd1ddc4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:10.088109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:10.130306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289a138-03cf-46cc-851d-431edbd42451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:10.130908 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289a138-03cf-46cc-851d-431edbd42451 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:10.131183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289a138-03cf-46cc-851d-431edbd42451 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:10.131450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289a138-03cf-46cc-851d-431edbd42451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:10.133068 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8289a138-03cf-46cc-851d-431edbd42451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:10.239434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289a138-03cf-46cc-851d-431edbd42451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:10.240842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289a138-03cf-46cc-851d-431edbd42451 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:10.259430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 733/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:12.648421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8913d-622f-4809-94b8-0486a8b2fb7b None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:12.648990 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8913d-622f-4809-94b8-0486a8b2fb7b None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:12.648990 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8913d-622f-4809-94b8-0486a8b2fb7b None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:12.650363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 711/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:12.655925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48271f-302c-4be9-9183-b244a4e35af8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:12.656349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48271f-302c-4be9-9183-b244a4e35af8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:12.656469 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48271f-302c-4be9-9183-b244a4e35af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:12.721377 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9d48271f-302c-4be9-9183-b244a4e35af8 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:12.722583 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9d48271f-302c-4be9-9183-b244a4e35af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:12.735729 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:12.818970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 734/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:13.100346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8913d-622f-4809-94b8-0486a8b2fb7b None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:13.101894 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-09b8913d-622f-4809-94b8-0486a8b2fb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:13.203216 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8913d-622f-4809-94b8-0486a8b2fb7b None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:13.216201 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:13.276504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b87978-3b94-4fa6-9a43-f871cf2e0b5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:13.277105 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b87978-3b94-4fa6-9a43-f871cf2e0b5f service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:13.277635 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b87978-3b94-4fa6-9a43-f871cf2e0b5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:13.277908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b87978-3b94-4fa6-9a43-f871cf2e0b5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:13.279492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b3b87978-3b94-4fa6-9a43-f871cf2e0b5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:13.385488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b87978-3b94-4fa6-9a43-f871cf2e0b5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:13.386934 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b87978-3b94-4fa6-9a43-f871cf2e0b5f service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:13.404869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 712/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:15.839069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3a57c-a221-4f71-a0ca-7a5baa758f62 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:15.839674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3a57c-a221-4f71-a0ca-7a5baa758f62 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:15.839674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3a57c-a221-4f71-a0ca-7a5baa758f62 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:15.840962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 735/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:15.846250 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f362b4-b333-4369-bcdd-108063b7feea None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:15.846684 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f362b4-b333-4369-bcdd-108063b7feea None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:15.846812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f362b4-b333-4369-bcdd-108063b7feea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:15.896549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:15.915077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-26f362b4-b333-4369-bcdd-108063b7feea None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:15.916336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-26f362b4-b333-4369-bcdd-108063b7feea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:16.010606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 713/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:16.295425 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f3a57c-a221-4f71-a0ca-7a5baa758f62 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:16.297043 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-95f3a57c-a221-4f71-a0ca-7a5baa758f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:16.404207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f3a57c-a221-4f71-a0ca-7a5baa758f62 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:16.443517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:16.477503 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b26bb-1770-477c-b28f-b12ba66eaec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:16.478106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b26bb-1770-477c-b28f-b12ba66eaec2 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:16.478387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b26bb-1770-477c-b28f-b12ba66eaec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:16.478662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b26bb-1770-477c-b28f-b12ba66eaec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:16.480284 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d16b26bb-1770-477c-b28f-b12ba66eaec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:16.588095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b26bb-1770-477c-b28f-b12ba66eaec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:16.589447 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b26bb-1770-477c-b28f-b12ba66eaec2 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:16.607200 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 736/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:18.866134 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f03a7-e2e1-4fac-94d7-a080b5c48be6 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:18.866678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f03a7-e2e1-4fac-94d7-a080b5c48be6 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:18.866678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f03a7-e2e1-4fac-94d7-a080b5c48be6 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:18.868076 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 714/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:18.873422 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78862339-ff3f-4ecc-84aa-eae627cf2da0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:18.873840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78862339-ff3f-4ecc-84aa-eae627cf2da0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:18.873971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78862339-ff3f-4ecc-84aa-eae627cf2da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:18.940218 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-78862339-ff3f-4ecc-84aa-eae627cf2da0 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:18.941448 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-78862339-ff3f-4ecc-84aa-eae627cf2da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:18.962005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:19.037665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 737/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:19.324708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f03a7-e2e1-4fac-94d7-a080b5c48be6 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:19.326244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-464f03a7-e2e1-4fac-94d7-a080b5c48be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:19.427846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f03a7-e2e1-4fac-94d7-a080b5c48be6 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:19.447628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:19.501497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840bdd7-5248-4423-aad5-2ad8b272798f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:19.502096 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840bdd7-5248-4423-aad5-2ad8b272798f service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:19.502375 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840bdd7-5248-4423-aad5-2ad8b272798f service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:19.502640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840bdd7-5248-4423-aad5-2ad8b272798f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:19.504299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-0840bdd7-5248-4423-aad5-2ad8b272798f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:19.612590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840bdd7-5248-4423-aad5-2ad8b272798f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:19.614615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840bdd7-5248-4423-aad5-2ad8b272798f service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:19.632430 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 715/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:21.828568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec28168-255e-44a5-899d-05ec0999d9a3 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:21.829126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec28168-255e-44a5-899d-05ec0999d9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:21.829126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec28168-255e-44a5-899d-05ec0999d9a3 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:21.830511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 738/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:21.836338 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba085d7d-80bc-4973-9d2f-b2c5eeefcd17 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:21.836792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba085d7d-80bc-4973-9d2f-b2c5eeefcd17 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:21.836896 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba085d7d-80bc-4973-9d2f-b2c5eeefcd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:21.898716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:21.901536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-ba085d7d-80bc-4973-9d2f-b2c5eeefcd17 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:21.903629 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ba085d7d-80bc-4973-9d2f-b2c5eeefcd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:21.998909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 716/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:22.286020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec28168-255e-44a5-899d-05ec0999d9a3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:22.287622 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bec28168-255e-44a5-899d-05ec0999d9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:22.398998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec28168-255e-44a5-899d-05ec0999d9a3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:22.447561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:22.471759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298cb3b-10e2-4dfe-9053-d4b8b8193f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:22.472363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298cb3b-10e2-4dfe-9053-d4b8b8193f99 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:22.472673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298cb3b-10e2-4dfe-9053-d4b8b8193f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:22.472950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298cb3b-10e2-4dfe-9053-d4b8b8193f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:22.474525 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a298cb3b-10e2-4dfe-9053-d4b8b8193f99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:22.581536 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298cb3b-10e2-4dfe-9053-d4b8b8193f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:22.583497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298cb3b-10e2-4dfe-9053-d4b8b8193f99 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:22.600909 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 739/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:24.799056 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace21ea-92ca-49e4-bae4-b1ad46a6dcfe None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:24.799643 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace21ea-92ca-49e4-bae4-b1ad46a6dcfe None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:24.799643 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace21ea-92ca-49e4-bae4-b1ad46a6dcfe None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:24.801651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 717/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:22:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:24.806886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abd29f-aae1-4499-b448-1c40a7b2e6ef None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:24.807291 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abd29f-aae1-4499-b448-1c40a7b2e6ef None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:24.807468 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abd29f-aae1-4499-b448-1c40a7b2e6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:24.874764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-77abd29f-aae1-4499-b448-1c40a7b2e6ef None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:24.876023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-77abd29f-aae1-4499-b448-1c40a7b2e6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:24.905046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:24.976978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 740/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:25.258571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace21ea-92ca-49e4-bae4-b1ad46a6dcfe None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:25.260180 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4ace21ea-92ca-49e4-bae4-b1ad46a6dcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:25.361050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace21ea-92ca-49e4-bae4-b1ad46a6dcfe None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:25.389333 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:25.434643 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf63aad-ddb2-4e0c-8915-8defcea92b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:25.435246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf63aad-ddb2-4e0c-8915-8defcea92b32 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:25.435526 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf63aad-ddb2-4e0c-8915-8defcea92b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:25.435834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf63aad-ddb2-4e0c-8915-8defcea92b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:25.437422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-edf63aad-ddb2-4e0c-8915-8defcea92b32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:25.543793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf63aad-ddb2-4e0c-8915-8defcea92b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:25.545155 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf63aad-ddb2-4e0c-8915-8defcea92b32 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:25.563113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 718/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:27.756714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31b42c-b74f-4919-bf36-418a5c08901d None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:27.757261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31b42c-b74f-4919-bf36-418a5c08901d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:27.757261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31b42c-b74f-4919-bf36-418a5c08901d None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:27.758605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 741/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:27.764107 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c3669-0bae-4330-a04c-3929e636a18b None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:27.764503 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c3669-0bae-4330-a04c-3929e636a18b None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:27.764682 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c3669-0bae-4330-a04c-3929e636a18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:27.830123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-d11c3669-0bae-4330-a04c-3929e636a18b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:27.831356 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d11c3669-0bae-4330-a04c-3929e636a18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:27.839446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:27.927980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 719/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:28.210202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31b42c-b74f-4919-bf36-418a5c08901d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:28.211761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ae31b42c-b74f-4919-bf36-418a5c08901d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:28.312080 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31b42c-b74f-4919-bf36-418a5c08901d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:28.366102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:28.383805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081197a5-8012-47e3-825c-2b09306c2fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:28.384404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081197a5-8012-47e3-825c-2b09306c2fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:28.384678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081197a5-8012-47e3-825c-2b09306c2fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:28.384950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081197a5-8012-47e3-825c-2b09306c2fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:28.386521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-081197a5-8012-47e3-825c-2b09306c2fa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:28.496049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081197a5-8012-47e3-825c-2b09306c2fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:28.497413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081197a5-8012-47e3-825c-2b09306c2fa2 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:28.631680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 742/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:30.796788 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9449d-b279-4750-8c8b-60d993556975 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:30.797373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9449d-b279-4750-8c8b-60d993556975 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:30.797373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9449d-b279-4750-8c8b-60d993556975 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:30.798732 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 720/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:30.804696 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3ebfc-eef5-4a05-8995-08b5b604965c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:30.805123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3ebfc-eef5-4a05-8995-08b5b604965c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:30.805230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3ebfc-eef5-4a05-8995-08b5b604965c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:30.868775 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c1d3ebfc-eef5-4a05-8995-08b5b604965c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:30.869964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c1d3ebfc-eef5-4a05-8995-08b5b604965c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:30.907259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:30.968044 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 743/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:31.251295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9449d-b279-4750-8c8b-60d993556975 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:31.252860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-0fb9449d-b279-4750-8c8b-60d993556975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:31.353927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9449d-b279-4750-8c8b-60d993556975 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:31.390315 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:31.425528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2db07-d6b2-401b-a037-c3bfdc69672d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:31.426708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2db07-d6b2-401b-a037-c3bfdc69672d service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:31.426982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2db07-d6b2-401b-a037-c3bfdc69672d service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:31.427249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2db07-d6b2-401b-a037-c3bfdc69672d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:31.428858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6ef2db07-d6b2-401b-a037-c3bfdc69672d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:31.533157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2db07-d6b2-401b-a037-c3bfdc69672d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:31.534532 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2db07-d6b2-401b-a037-c3bfdc69672d service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:31.553227 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 721/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:33.741031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704b680-1191-483e-aa74-e821f1e885ac None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:33.741594 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704b680-1191-483e-aa74-e821f1e885ac None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:33.741594 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704b680-1191-483e-aa74-e821f1e885ac None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:33.743025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 744/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:33.748233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f3f5a-db26-4a4f-b73e-ff30507d1276 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:33.748638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f3f5a-db26-4a4f-b73e-ff30507d1276 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:33.748782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f3f5a-db26-4a4f-b73e-ff30507d1276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:33.815275 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-684f3f5a-db26-4a4f-b73e-ff30507d1276 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:33.817022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-684f3f5a-db26-4a4f-b73e-ff30507d1276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:33.835438 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:33.912276 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 722/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:34.198998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704b680-1191-483e-aa74-e821f1e885ac None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:34.200561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5704b680-1191-483e-aa74-e821f1e885ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:34.302708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704b680-1191-483e-aa74-e821f1e885ac None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:34.371466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:34.378261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95d3a8-3c01-426a-83d7-b555f6fef50a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:34.378880 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95d3a8-3c01-426a-83d7-b555f6fef50a service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:34.379142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95d3a8-3c01-426a-83d7-b555f6fef50a service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:34.379413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95d3a8-3c01-426a-83d7-b555f6fef50a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:34.381066 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1f95d3a8-3c01-426a-83d7-b555f6fef50a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:34.487944 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95d3a8-3c01-426a-83d7-b555f6fef50a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:34.489362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95d3a8-3c01-426a-83d7-b555f6fef50a service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:34.507111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 745/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:36.701408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a95884-9d87-4343-bf7a-001581c07a71 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:36.701977 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a95884-9d87-4343-bf7a-001581c07a71 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:36.702104 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a95884-9d87-4343-bf7a-001581c07a71 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:36.703396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 723/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:36.708824 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f496e-4575-4781-a083-f7b1802b78b0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:36.709218 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f496e-4575-4781-a083-f7b1802b78b0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:36.709365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f496e-4575-4781-a083-f7b1802b78b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:36.773299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d62f496e-4575-4781-a083-f7b1802b78b0 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:36.774484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d62f496e-4575-4781-a083-f7b1802b78b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:36.821921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:36.869454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 746/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:37.154251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a95884-9d87-4343-bf7a-001581c07a71 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:37.155851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-17a95884-9d87-4343-bf7a-001581c07a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:37.256642 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a95884-9d87-4343-bf7a-001581c07a71 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:37.304914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:37.329926 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb1d46-c226-4688-857a-a99aa48c63af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:37.330523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb1d46-c226-4688-857a-a99aa48c63af service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:37.330798 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb1d46-c226-4688-857a-a99aa48c63af service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:37.331062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb1d46-c226-4688-857a-a99aa48c63af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:37.332669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-caeb1d46-c226-4688-857a-a99aa48c63af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:37.441311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb1d46-c226-4688-857a-a99aa48c63af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:37.442776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb1d46-c226-4688-857a-a99aa48c63af service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:37.460473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 724/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:22:46.868007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=22 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:47.458218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=66 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:57.350965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453be21-0a42-4379-b690-8899b10a56f7 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:57.351519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453be21-0a42-4379-b690-8899b10a56f7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:57.351519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453be21-0a42-4379-b690-8899b10a56f7 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:57.352872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 747/1471] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:57.358250 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f519f-3e79-4320-8d48-e223dcd8d1ae None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:57.358714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f519f-3e79-4320-8d48-e223dcd8d1ae None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:57.358829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f519f-3e79-4320-8d48-e223dcd8d1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:57.425806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b55f519f-3e79-4320-8d48-e223dcd8d1ae None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:57.427015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b55f519f-3e79-4320-8d48-e223dcd8d1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:57.521323 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 725/1472] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:57.821417 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bfb29-8d0b-4d38-8d2a-bba1c455c824 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:57.821698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bfb29-8d0b-4d38-8d2a-bba1c455c824 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:57.821971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bfb29-8d0b-4d38-8d2a-bba1c455c824 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:57.823315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 748/1473] 10.222.0.27 () {54 vars in 966 bytes} [Mon Sep 12 23:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:22:57.828217 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1f12-c6c0-4a91-aceb-589302edf42c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:57.828503 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1f12-c6c0-4a91-aceb-589302edf42c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:57.828776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1f12-c6c0-4a91-aceb-589302edf42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:57.930345 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-d40e1f12-c6c0-4a91-aceb-589302edf42c None None] MFA Rules not processed for user `d61407406ef242a78030f79532fe2247`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:22:57.931987 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d40e1f12-c6c0-4a91-aceb-589302edf42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:57.954077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:58.008556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 726/1474] 10.222.0.27 () {58 vars in 1063 bytes} [Mon Sep 12 23:22:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:22:58.016358 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bfb29-8d0b-4d38-8d2a-bba1c455c824 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:22:58.017872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8e1bfb29-8d0b-4d38-8d2a-bba1c455c824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:58.118098 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bfb29-8d0b-4d38-8d2a-bba1c455c824 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:22:58.189916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74f2cb-a0f5-4fa8-9b28-2a3383f7da5c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61407406ef242a78030f79532fe2247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:22:58.190441 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74f2cb-a0f5-4fa8-9b28-2a3383f7da5c service glance] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:22:58.190716 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74f2cb-a0f5-4fa8-9b28-2a3383f7da5c service glance] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:22:58.190986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74f2cb-a0f5-4fa8-9b28-2a3383f7da5c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:22:58.193142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-da74f2cb-a0f5-4fa8-9b28-2a3383f7da5c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:22:58.250737 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:22:58.306437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74f2cb-a0f5-4fa8-9b28-2a3383f7da5c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:22:58.308022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74f2cb-a0f5-4fa8-9b28-2a3383f7da5c service glance] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:22:58.325977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 749/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:23:00.549063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c731b-2898-4d4f-842c-be65e2139d6a None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:00.549678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c731b-2898-4d4f-842c-be65e2139d6a None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:00.549678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c731b-2898-4d4f-842c-be65e2139d6a None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:00.551622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 727/1476] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:23:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:23:00.556930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a345e3a-89f4-40fe-a454-842a418d8356 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:00.557386 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a345e3a-89f4-40fe-a454-842a418d8356 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:00.557500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a345e3a-89f4-40fe-a454-842a418d8356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:00.622034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2a345e3a-89f4-40fe-a454-842a418d8356 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:23:00.623219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2a345e3a-89f4-40fe-a454-842a418d8356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:00.717220 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 750/1477] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:23:00.988961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c731b-2898-4d4f-842c-be65e2139d6a None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:23:00.990500 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3a3c731b-2898-4d4f-842c-be65e2139d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:01.112175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c731b-2898-4d4f-842c-be65e2139d6a None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:23:01.160069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:23:01.203949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b904f0-c561-46be-a923-fc88bd727ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:23:01.204484 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b904f0-c561-46be-a923-fc88bd727ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:01.204760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b904f0-c561-46be-a923-fc88bd727ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:01.205029 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b904f0-c561-46be-a923-fc88bd727ff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:01.205657 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b904f0-c561-46be-a923-fc88bd727ff3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:23:01.207417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: WARNING py.warnings [None req-b0b904f0-c561-46be-a923-fc88bd727ff3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:23:01.207417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: warnings.warn(msg) Sep 12 23:23:01.207417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: Sep 12 23:23:01.208589 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b904f0-c561-46be-a923-fc88bd727ff3 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:23:01.227076 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 728/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Sep 12 23:23:00 2022] GET /identity/v3/projects => generated 2553 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:23:03.382248 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be315602-a691-472f-933e-3d35f94a04db None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:03.382781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be315602-a691-472f-933e-3d35f94a04db None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:03.382781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be315602-a691-472f-933e-3d35f94a04db None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:03.384092 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 751/1479] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:23:03.389399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f75ace-b496-4634-872a-37c25ad539f6 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:03.389844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f75ace-b496-4634-872a-37c25ad539f6 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:03.389954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f75ace-b496-4634-872a-37c25ad539f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:03.455633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-08f75ace-b496-4634-872a-37c25ad539f6 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:23:03.457365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-08f75ace-b496-4634-872a-37c25ad539f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:03.551369 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 729/1480] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:23:03.848566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be315602-a691-472f-933e-3d35f94a04db None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:23:03.850034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-be315602-a691-472f-933e-3d35f94a04db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:03.859748 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:23:03.994063 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be315602-a691-472f-933e-3d35f94a04db None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:23:04.073089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:23:04.104438 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054438f-e4fc-4c1c-8f47-634f33a8e7e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:23:04.105050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054438f-e4fc-4c1c-8f47-634f33a8e7e9 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:04.105327 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054438f-e4fc-4c1c-8f47-634f33a8e7e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:04.105601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054438f-e4fc-4c1c-8f47-634f33a8e7e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:04.107183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a054438f-e4fc-4c1c-8f47-634f33a8e7e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:04.195655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a054438f-e4fc-4c1c-8f47-634f33a8e7e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:23:04.197216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a054438f-e4fc-4c1c-8f47-634f33a8e7e9 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:23:04.214316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 752/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:23:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:23:06.521709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51572c26-977b-4761-80bb-706e60a66d1e None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:06.522253 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51572c26-977b-4761-80bb-706e60a66d1e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:06.522253 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51572c26-977b-4761-80bb-706e60a66d1e None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:06.523618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 730/1482] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:23:06.528767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fea89-a14c-4ece-b509-04243ccea85d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:06.529432 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fea89-a14c-4ece-b509-04243ccea85d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:06.529572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fea89-a14c-4ece-b509-04243ccea85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:06.593960 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-de5fea89-a14c-4ece-b509-04243ccea85d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:23:06.595307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-de5fea89-a14c-4ece-b509-04243ccea85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:06.644296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:23:06.695284 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 753/1483] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:23:06.979754 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51572c26-977b-4761-80bb-706e60a66d1e None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:23:06.981308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-51572c26-977b-4761-80bb-706e60a66d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:07.006852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:23:07.085021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51572c26-977b-4761-80bb-706e60a66d1e None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:23:07.157218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c90e541-0750-490a-8d0c-7dd7abbb0353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:23:07.157824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90e541-0750-490a-8d0c-7dd7abbb0353 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:07.158110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90e541-0750-490a-8d0c-7dd7abbb0353 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:07.158384 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90e541-0750-490a-8d0c-7dd7abbb0353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:07.160058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7c90e541-0750-490a-8d0c-7dd7abbb0353 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:07.231807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:23:07.266189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c90e541-0750-490a-8d0c-7dd7abbb0353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:23:07.267654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c90e541-0750-490a-8d0c-7dd7abbb0353 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:23:07.286019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 731/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:23:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:23:09.481383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf71bab-9b58-41df-bf03-bbb2d91035eb None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:09.481945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf71bab-9b58-41df-bf03-bbb2d91035eb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:09.481945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf71bab-9b58-41df-bf03-bbb2d91035eb None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:09.483267 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 754/1485] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:23:09.489049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6121c68-54a5-4578-9c8d-f1b1293353fa None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:09.489459 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6121c68-54a5-4578-9c8d-f1b1293353fa None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:09.489606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6121c68-54a5-4578-9c8d-f1b1293353fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:09.556710 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b6121c68-54a5-4578-9c8d-f1b1293353fa None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:23:09.557925 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b6121c68-54a5-4578-9c8d-f1b1293353fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:09.652120 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 732/1486] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:23:09.944637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf71bab-9b58-41df-bf03-bbb2d91035eb None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:23:09.946179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-baf71bab-9b58-41df-bf03-bbb2d91035eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:10.047322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf71bab-9b58-41df-bf03-bbb2d91035eb None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:23:10.118379 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:23:10.119934 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf26f9-4e42-44d7-ba98-2898847f50fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:23:10.120525 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf26f9-4e42-44d7-ba98-2898847f50fd service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:10.120983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf26f9-4e42-44d7-ba98-2898847f50fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:10.121502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf26f9-4e42-44d7-ba98-2898847f50fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:10.123249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c0cf26f9-4e42-44d7-ba98-2898847f50fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:10.230826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf26f9-4e42-44d7-ba98-2898847f50fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:23:10.232221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf26f9-4e42-44d7-ba98-2898847f50fd service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:23:10.250460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 755/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:23:12.482000 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b921f8b-594f-45ea-996a-73616b75aee3 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:12.482536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b921f8b-594f-45ea-996a-73616b75aee3 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:12.482536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b921f8b-594f-45ea-996a-73616b75aee3 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:12.483917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 733/1488] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:23:12.489105 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f4ffe-4b7b-423f-b0b8-a579459df043 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:12.489515 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f4ffe-4b7b-423f-b0b8-a579459df043 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:12.489676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f4ffe-4b7b-423f-b0b8-a579459df043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:12.553886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-fc4f4ffe-4b7b-423f-b0b8-a579459df043 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:23:12.555086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fc4f4ffe-4b7b-423f-b0b8-a579459df043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:12.606644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:23:12.650062 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 756/1489] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:23:12.938211 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b921f8b-594f-45ea-996a-73616b75aee3 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:23:12.939820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-0b921f8b-594f-45ea-996a-73616b75aee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:12.975123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:23:13.046503 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b921f8b-594f-45ea-996a-73616b75aee3 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:23:13.119024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671641af-da94-41c8-970d-bd13ea72215e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:23:13.119660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671641af-da94-41c8-970d-bd13ea72215e service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:13.119940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671641af-da94-41c8-970d-bd13ea72215e service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:13.120229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671641af-da94-41c8-970d-bd13ea72215e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:13.121812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-671641af-da94-41c8-970d-bd13ea72215e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:13.200231 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:23:13.229665 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671641af-da94-41c8-970d-bd13ea72215e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:23:13.231110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671641af-da94-41c8-970d-bd13ea72215e service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:23:13.248722 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 734/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:23:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:23:15.590225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6c88a-d012-4283-9927-c0d5b6073e88 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:15.590799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6c88a-d012-4283-9927-c0d5b6073e88 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:15.590799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6c88a-d012-4283-9927-c0d5b6073e88 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:15.592154 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 757/1491] 10.222.0.27 () {54 vars in 927 bytes} [Mon Sep 12 23:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:23:15.597906 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170d287-e3bb-466d-8e29-3eb9fa27dddf None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:15.598311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170d287-e3bb-466d-8e29-3eb9fa27dddf None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:15.598459 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170d287-e3bb-466d-8e29-3eb9fa27dddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:15.666515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-7170d287-e3bb-466d-8e29-3eb9fa27dddf None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:23:15.667758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7170d287-e3bb-466d-8e29-3eb9fa27dddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:15.763091 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 735/1492] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Sep 12 23:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:23:16.058060 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6c88a-d012-4283-9927-c0d5b6073e88 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:23:16.059611 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5be6c88a-d012-4283-9927-c0d5b6073e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:16.159261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6c88a-d012-4283-9927-c0d5b6073e88 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:23:16.230629 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ba9c7-1968-4bd8-82d6-b8b4840cdfe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:23:16.231161 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ba9c7-1968-4bd8-82d6-b8b4840cdfe6 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:23:16.231439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ba9c7-1968-4bd8-82d6-b8b4840cdfe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:23:16.231745 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ba9c7-1968-4bd8-82d6-b8b4840cdfe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:23:16.233326 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9d4ba9c7-1968-4bd8-82d6-b8b4840cdfe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:23:16.241721 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:23:16.340164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ba9c7-1968-4bd8-82d6-b8b4840cdfe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:23:16.341582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ba9c7-1968-4bd8-82d6-b8b4840cdfe6 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:23:16.359704 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 758/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:23:25.761589 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=82 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:23:26.357500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=50 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:24:35.490880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9f484-68b6-4a27-bcda-29428c07b740 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:35.491476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9f484-68b6-4a27-bcda-29428c07b740 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:35.491476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9f484-68b6-4a27-bcda-29428c07b740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:35.575767 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-cbf9f484-68b6-4a27-bcda-29428c07b740 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:24:35.576973 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-cbf9f484-68b6-4a27-bcda-29428c07b740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:35.788220 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 736/1494] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:24:35.796371 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115eebf-2026-4168-a998-25b17dd2ac72 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:35.796866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115eebf-2026-4168-a998-25b17dd2ac72 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:35.796971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115eebf-2026-4168-a998-25b17dd2ac72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:35.883399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0115eebf-2026-4168-a998-25b17dd2ac72 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:24:35.884641 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0115eebf-2026-4168-a998-25b17dd2ac72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:35.979882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 759/1495] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:24:35.984459 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9f484-68b6-4a27-bcda-29428c07b740 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:24:35.985654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-cbf9f484-68b6-4a27-bcda-29428c07b740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:35.997372 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9f484-68b6-4a27-bcda-29428c07b740 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:24:36.057965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:24:36.089929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81168f-e146-4120-9a6d-0fe9acb7ac53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:24:36.090549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81168f-e146-4120-9a6d-0fe9acb7ac53 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:36.090813 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81168f-e146-4120-9a6d-0fe9acb7ac53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:36.091088 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81168f-e146-4120-9a6d-0fe9acb7ac53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:36.091757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81168f-e146-4120-9a6d-0fe9acb7ac53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:24:36.092064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81168f-e146-4120-9a6d-0fe9acb7ac53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:24:36.093084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: WARNING py.warnings [None req-1f81168f-e146-4120-9a6d-0fe9acb7ac53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:24:36.093084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: warnings.warn(msg) Sep 12 23:24:36.093084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: Sep 12 23:24:36.094006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81168f-e146-4120-9a6d-0fe9acb7ac53 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:24:36.106092 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 737/1496] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:24:36.114981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115eebf-2026-4168-a998-25b17dd2ac72 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:24:36.124011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115eebf-2026-4168-a998-25b17dd2ac72 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:24:36.188081 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:24:36.223806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db90297-4b1d-46be-b275-c2eeee27ddd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:24:36.224413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db90297-4b1d-46be-b275-c2eeee27ddd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:36.224692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db90297-4b1d-46be-b275-c2eeee27ddd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:36.224971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db90297-4b1d-46be-b275-c2eeee27ddd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:36.225600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db90297-4b1d-46be-b275-c2eeee27ddd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:24:36.227339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-9db90297-4b1d-46be-b275-c2eeee27ddd8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:24:36.227339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 12 23:24:36.227339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: Sep 12 23:24:36.228634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db90297-4b1d-46be-b275-c2eeee27ddd8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:24:36.282884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 760/1497] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:24:36 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:24:36.287560 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81168f-e146-4120-9a6d-0fe9acb7ac53 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:24:36.289156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1f81168f-e146-4120-9a6d-0fe9acb7ac53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:36.322042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81168f-e146-4120-9a6d-0fe9acb7ac53 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:24:36.413362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17506156-560c-4955-af88-b7e90b6bdf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:24:36.413964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17506156-560c-4955-af88-b7e90b6bdf56 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:36.414240 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17506156-560c-4955-af88-b7e90b6bdf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:36.414510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17506156-560c-4955-af88-b7e90b6bdf56 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:36.415135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17506156-560c-4955-af88-b7e90b6bdf56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:24:36.416181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: WARNING py.warnings [None req-17506156-560c-4955-af88-b7e90b6bdf56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:24:36.416181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: warnings.warn(msg) Sep 12 23:24:36.416181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: Sep 12 23:24:36.417373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17506156-560c-4955-af88-b7e90b6bdf56 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:24:36.458241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 738/1498] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:24:36 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:24:36.462366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db90297-4b1d-46be-b275-c2eeee27ddd8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:24:36.463886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9db90297-4b1d-46be-b275-c2eeee27ddd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:36.525355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db90297-4b1d-46be-b275-c2eeee27ddd8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:24:36.610534 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:24:36.623065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9a0c9-964c-429d-b1d2-a0e81bbdb482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:24:36.623698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9a0c9-964c-429d-b1d2-a0e81bbdb482 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:36.623975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9a0c9-964c-429d-b1d2-a0e81bbdb482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:36.624244 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9a0c9-964c-429d-b1d2-a0e81bbdb482 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:36.624879 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9a0c9-964c-429d-b1d2-a0e81bbdb482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:24:36.625879 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-dfd9a0c9-964c-429d-b1d2-a0e81bbdb482 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:24:36.625879 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 12 23:24:36.625879 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: Sep 12 23:24:36.626758 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9a0c9-964c-429d-b1d2-a0e81bbdb482 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:24:36.642283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 761/1499] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:24:36.644298 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17506156-560c-4955-af88-b7e90b6bdf56 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:24:36.645816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-17506156-560c-4955-af88-b7e90b6bdf56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:36.657227 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17506156-560c-4955-af88-b7e90b6bdf56 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:24:36.681579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:24:36.750688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d948c2-75ea-49b6-a0bb-0e9370ae7a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:24:36.751314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d948c2-75ea-49b6-a0bb-0e9370ae7a3e admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:36.751650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d948c2-75ea-49b6-a0bb-0e9370ae7a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:36.751924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d948c2-75ea-49b6-a0bb-0e9370ae7a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:36.753061 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d948c2-75ea-49b6-a0bb-0e9370ae7a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:24:36.754072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: WARNING py.warnings [None req-a2d948c2-75ea-49b6-a0bb-0e9370ae7a3e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:24:36.754072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: warnings.warn(msg) Sep 12 23:24:36.754072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: Sep 12 23:24:36.754952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d948c2-75ea-49b6-a0bb-0e9370ae7a3e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:24:36.769386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 739/1500] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:24:36.773592 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9a0c9-964c-429d-b1d2-a0e81bbdb482 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:24:36.775132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dfd9a0c9-964c-429d-b1d2-a0e81bbdb482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:36.807977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9a0c9-964c-429d-b1d2-a0e81bbdb482 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:24:36.899058 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4e138-7355-4fd4-bc44-8e03841cb323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:24:36.899657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4e138-7355-4fd4-bc44-8e03841cb323 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:36.899937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4e138-7355-4fd4-bc44-8e03841cb323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:36.900204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4e138-7355-4fd4-bc44-8e03841cb323 admin admin] PATH_INFO: `/v3/projects/510fb82d7804465a81cb86f1215aace4/users/836837b09b4f4558a3006ab581cc1a37/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:36.922814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4e138-7355-4fd4-bc44-8e03841cb323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=510fb82d7804465a81cb86f1215aace4, user_id=836837b09b4f4558a3006ab581cc1a37, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:24:36.924373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-70b4e138-7355-4fd4-bc44-8e03841cb323 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:24:36.924373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 12 23:24:36.924373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: Sep 12 23:24:36.926503 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4e138-7355-4fd4-bc44-8e03841cb323 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:24:36.950708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 762/1501] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:24:36 2022] PUT /identity/v3/projects/510fb82d7804465a81cb86f1215aace4/users/836837b09b4f4558a3006ab581cc1a37/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:24:36.962756 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f9a25-3115-4f48-8207-b53a39e77e26 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:36.963037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f9a25-3115-4f48-8207-b53a39e77e26 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:36.963305 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f9a25-3115-4f48-8207-b53a39e77e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:37.018253 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-eb2f9a25-3115-4f48-8207-b53a39e77e26 None None] MFA Rules not processed for user `836837b09b4f4558a3006ab581cc1a37`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:24:37.019445 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-eb2f9a25-3115-4f48-8207-b53a39e77e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:37.092876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 740/1502] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:24:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:24:37.105820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850ff8f-5298-4ee5-aa6c-f048b5e9c065 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:37.106140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850ff8f-5298-4ee5-aa6c-f048b5e9c065 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:37.106418 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850ff8f-5298-4ee5-aa6c-f048b5e9c065 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:37.107843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 763/1503] 10.222.0.27 () {54 vars in 964 bytes} [Mon Sep 12 23:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:24:37.112944 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c8513-138d-4175-80ee-e3af2b96d339 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:37.113234 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c8513-138d-4175-80ee-e3af2b96d339 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:37.113512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c8513-138d-4175-80ee-e3af2b96d339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:37.144315 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:24:37.191844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-4b2c8513-138d-4175-80ee-e3af2b96d339 None None] MFA Rules not processed for user `c5b8828077594349a93c7a842c0f15da`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:24:37.193083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4b2c8513-138d-4175-80ee-e3af2b96d339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:37.266871 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 741/1504] 10.222.0.27 () {58 vars in 1061 bytes} [Mon Sep 12 23:24:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:24:37.274620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850ff8f-5298-4ee5-aa6c-f048b5e9c065 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:24:37.276176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4850ff8f-5298-4ee5-aa6c-f048b5e9c065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:37.325880 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:24:37.380005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850ff8f-5298-4ee5-aa6c-f048b5e9c065 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:24:37.455977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c78fe0-7e89-4b55-9367-01f77ded7545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:24:37.456514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c78fe0-7e89-4b55-9367-01f77ded7545 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:37.456792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c78fe0-7e89-4b55-9367-01f77ded7545 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:37.457091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c78fe0-7e89-4b55-9367-01f77ded7545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:37.458701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-93c78fe0-7e89-4b55-9367-01f77ded7545 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:37.557349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c78fe0-7e89-4b55-9367-01f77ded7545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:24:37.558790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c78fe0-7e89-4b55-9367-01f77ded7545 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:24:37.562187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:24:37.576768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 764/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:24:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:24:37.623488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7016a-57f3-4f1a-bfc2-6fa33bb1e85d None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:37.623811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7016a-57f3-4f1a-bfc2-6fa33bb1e85d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:37.624093 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7016a-57f3-4f1a-bfc2-6fa33bb1e85d None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:37.625387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 742/1506] 10.222.0.27 () {52 vars in 830 bytes} [Mon Sep 12 23:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:24:37.629000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c78fe0-7e89-4b55-9367-01f77ded7545 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:24:37.630232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-93c78fe0-7e89-4b55-9367-01f77ded7545 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:37.726962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c78fe0-7e89-4b55-9367-01f77ded7545 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:24:37.817916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:24:37.818523 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:37.818797 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:37.819073 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] PATH_INFO: `/v3/users/836837b09b4f4558a3006ab581cc1a37` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:37.832007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=836837b09b4f4558a3006ab581cc1a37)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:24:37.833009 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:24:37.833009 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 12 23:24:37.833009 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: Sep 12 23:24:37.834203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:24:37.843480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:24:37.898749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '836837b09b4f4558a3006ab581cc1a37'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:24:37.914456 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '836837b09b4f4558a3006ab581cc1a37'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:24:37.937175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '836837b09b4f4558a3006ab581cc1a37'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:24:37.946393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '836837b09b4f4558a3006ab581cc1a37'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:24:37.946914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '836837b09b4f4558a3006ab581cc1a37'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:24:37.952350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 765/1507] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:24:37 2022] DELETE /identity/v3/users/836837b09b4f4558a3006ab581cc1a37 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:24:37.957310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208cc32-ba8d-4e86-8378-0359c5648ae8 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:37.957770 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208cc32-ba8d-4e86-8378-0359c5648ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:37.957879 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208cc32-ba8d-4e86-8378-0359c5648ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:38.012868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-6208cc32-ba8d-4e86-8378-0359c5648ae8 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:24:38.013985 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6208cc32-ba8d-4e86-8378-0359c5648ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:38.088995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:24:38.101937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 743/1508] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:24:38.111593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:24:38.112830 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:38.213879 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0dad89-e3f6-4326-ba94-f4aae26ddda4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:24:38.269162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:24:38.289842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb4544-84a5-426f-be56-df7207204265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:24:38.290456 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb4544-84a5-426f-be56-df7207204265 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:38.290790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb4544-84a5-426f-be56-df7207204265 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:38.291070 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb4544-84a5-426f-be56-df7207204265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:38.292726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e3fb4544-84a5-426f-be56-df7207204265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:38.402083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb4544-84a5-426f-be56-df7207204265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:24:38.403514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb4544-84a5-426f-be56-df7207204265 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:24:38.421190 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 766/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:24:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:24:38.768503 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208cc32-ba8d-4e86-8378-0359c5648ae8 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:24:38.769782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6208cc32-ba8d-4e86-8378-0359c5648ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:24:38.781669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208cc32-ba8d-4e86-8378-0359c5648ae8 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:24:38.884943 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:24:38.885575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:24:38.885854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:24:38.886126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9 admin admin] PATH_INFO: `/v3/projects/510fb82d7804465a81cb86f1215aace4` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:24:38.896399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=510fb82d7804465a81cb86f1215aace4)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:24:38.897435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: WARNING py.warnings [None req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:24:38.897435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: warnings.warn(msg) Sep 12 23:24:38.897435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: Sep 12 23:24:38.898605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:24:38.956202 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '510fb82d7804465a81cb86f1215aace4', 'request_id': 'req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:24:38.962114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '510fb82d7804465a81cb86f1215aace4', 'request_id': 'req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:24:38.972415 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '510fb82d7804465a81cb86f1215aace4', 'request_id': 'req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:24:39.001283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9 admin admin] The token cache is being invalidate because project 510fb82d7804465a81cb86f1215aace4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:24:39.001572 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:24:39.002628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-c892631a-a2cc-42bc-b84e-94e0c8b0f3f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:24:39.004018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 744/1510] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:24:38 2022] DELETE /identity/v3/projects/510fb82d7804465a81cb86f1215aace4 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:24:48.418971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=62 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:24:48.999134 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:26:01.949959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef3957-6cb4-46ba-8457-37163b8478f4 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:01.950472 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef3957-6cb4-46ba-8457-37163b8478f4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:01.950580 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef3957-6cb4-46ba-8457-37163b8478f4 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:01.951881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 767/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:26:01.956769 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c126ab-aa1f-4a0d-94f6-ad19b084b0bb None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:01.957211 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c126ab-aa1f-4a0d-94f6-ad19b084b0bb None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:01.957314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c126ab-aa1f-4a0d-94f6-ad19b084b0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:02.036558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-32c126ab-aa1f-4a0d-94f6-ad19b084b0bb None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:26:02.038208 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-32c126ab-aa1f-4a0d-94f6-ad19b084b0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:26:02.145476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 745/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:26:02.299472 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef3957-6cb4-46ba-8457-37163b8478f4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:26:02.301629 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-61ef3957-6cb4-46ba-8457-37163b8478f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:26:02.426948 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef3957-6cb4-46ba-8457-37163b8478f4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:26:02.432462 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a285e-a425-46db-aa80-e7c741e2b412 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:26:02.433072 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a285e-a425-46db-aa80-e7c741e2b412 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:02.433353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a285e-a425-46db-aa80-e7c741e2b412 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:02.433657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a285e-a425-46db-aa80-e7c741e2b412 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:02.441633 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a285e-a425-46db-aa80-e7c741e2b412 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:26:02.443206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a285e-a425-46db-aa80-e7c741e2b412 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:26:02.452415 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 768/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:26:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:26:03.548474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dfe94-318d-4795-aad5-a65202e13537 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:03.549000 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dfe94-318d-4795-aad5-a65202e13537 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:03.549000 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dfe94-318d-4795-aad5-a65202e13537 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:03.550269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 746/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:26:03.557905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b2e8b-e635-4495-b8d1-c5c8fd3158d2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:03.558397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b2e8b-e635-4495-b8d1-c5c8fd3158d2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:03.558518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b2e8b-e635-4495-b8d1-c5c8fd3158d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:03.577626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-977b2e8b-e635-4495-b8d1-c5c8fd3158d2 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:26:03.578809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-977b2e8b-e635-4495-b8d1-c5c8fd3158d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:26:03.586426 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 769/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:26:03.736778 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dfe94-318d-4795-aad5-a65202e13537 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:26:03.751672 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dfe94-318d-4795-aad5-a65202e13537 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:26:03.756915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53130b-9d9e-4153-9940-53edc8bda2bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:26:03.757497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53130b-9d9e-4153-9940-53edc8bda2bc service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:03.757780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53130b-9d9e-4153-9940-53edc8bda2bc service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:03.758042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53130b-9d9e-4153-9940-53edc8bda2bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:03.765776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53130b-9d9e-4153-9940-53edc8bda2bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:26:03.767358 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53130b-9d9e-4153-9940-53edc8bda2bc service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:26:03.777002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 747/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:26:04.087233 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313983de-2dc7-44e3-962e-1d52770dad28 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:04.087423 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313983de-2dc7-44e3-962e-1d52770dad28 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:04.087736 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313983de-2dc7-44e3-962e-1d52770dad28 None None] PATH_INFO: `/v3` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:04.089035 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 770/1517] 10.222.0.25 () {54 vars in 925 bytes} [Mon Sep 12 23:26:04 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:26:04.095300 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4b328-a053-4814-82df-40c6331a3b86 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:04.095617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4b328-a053-4814-82df-40c6331a3b86 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:04.095901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4b328-a053-4814-82df-40c6331a3b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:04.154468 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-36a4b328-a053-4814-82df-40c6331a3b86 None None] MFA Rules not processed for user `2b3b0a0317b14e29a32eb6ba6b89c2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:26:04.155700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-36a4b328-a053-4814-82df-40c6331a3b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:26:04.172131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:26:04.225497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 748/1518] 10.222.0.25 () {58 vars in 1015 bytes} [Mon Sep 12 23:26:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:26:07.743866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cc8d5-2c62-44de-a42b-698d0048cb5a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:07.744331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cc8d5-2c62-44de-a42b-698d0048cb5a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:07.744440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cc8d5-2c62-44de-a42b-698d0048cb5a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:07.745715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 771/1519] 10.222.0.27 () {54 vars in 961 bytes} [Mon Sep 12 23:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:26:07.750778 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c15c4-5d33-4e5d-a804-41bda49b0248 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:07.751272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c15c4-5d33-4e5d-a804-41bda49b0248 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:07.751382 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c15c4-5d33-4e5d-a804-41bda49b0248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:07.771193 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-643c15c4-5d33-4e5d-a804-41bda49b0248 None None] MFA Rules not processed for user `2b3b0a0317b14e29a32eb6ba6b89c2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:26:07.772621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-643c15c4-5d33-4e5d-a804-41bda49b0248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:26:07.781065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 749/1520] 10.222.0.27 () {58 vars in 1058 bytes} [Mon Sep 12 23:26:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:26:07.788901 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cc8d5-2c62-44de-a42b-698d0048cb5a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:26:07.801859 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cc8d5-2c62-44de-a42b-698d0048cb5a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:26:07.807269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1dd95-e007-4d5a-9bfd-553a49b39531 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b3b0a0317b14e29a32eb6ba6b89c2aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:26:07.807886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1dd95-e007-4d5a-9bfd-553a49b39531 service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:07.808163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1dd95-e007-4d5a-9bfd-553a49b39531 service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:07.808436 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1dd95-e007-4d5a-9bfd-553a49b39531 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:07.816685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1dd95-e007-4d5a-9bfd-553a49b39531 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:26:07.818947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1dd95-e007-4d5a-9bfd-553a49b39531 service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:26:07.828669 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 772/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:26:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:26:15.260439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3dbf2-fc6b-4433-b5b4-5f3e9e3c546b None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:15.260996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3dbf2-fc6b-4433-b5b4-5f3e9e3c546b None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:15.260996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3dbf2-fc6b-4433-b5b4-5f3e9e3c546b None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:15.262252 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 750/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:26:15.268326 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbe8e6-1a43-4dce-8950-7aed8501431d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:15.268740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbe8e6-1a43-4dce-8950-7aed8501431d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:15.268877 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbe8e6-1a43-4dce-8950-7aed8501431d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:15.288207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c0bbe8e6-1a43-4dce-8950-7aed8501431d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:26:15.289399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c0bbe8e6-1a43-4dce-8950-7aed8501431d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:26:15.297222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 773/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:26:15.451690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3dbf2-fc6b-4433-b5b4-5f3e9e3c546b None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:26:15.465974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3dbf2-fc6b-4433-b5b4-5f3e9e3c546b None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:26:15.471264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4fd44-8d77-4732-a920-3d5d1c8f0d30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:26:15.471902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4fd44-8d77-4732-a920-3d5d1c8f0d30 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:15.472179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4fd44-8d77-4732-a920-3d5d1c8f0d30 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:15.472448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4fd44-8d77-4732-a920-3d5d1c8f0d30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:15.480836 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4fd44-8d77-4732-a920-3d5d1c8f0d30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:26:15.482527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4fd44-8d77-4732-a920-3d5d1c8f0d30 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:26:15.492717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 751/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:26:16.574467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4ebf5-f2ff-4f28-9733-3be3a36c9607 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:16.575058 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4ebf5-f2ff-4f28-9733-3be3a36c9607 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:16.575058 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4ebf5-f2ff-4f28-9733-3be3a36c9607 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:16.576454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 774/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:26:16.582573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c763b8-24ad-477e-918a-21bf359d14c6 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:16.583031 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c763b8-24ad-477e-918a-21bf359d14c6 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:16.583141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c763b8-24ad-477e-918a-21bf359d14c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:16.602086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-94c763b8-24ad-477e-918a-21bf359d14c6 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:26:16.603277 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-94c763b8-24ad-477e-918a-21bf359d14c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:26:16.610979 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 752/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:26:16.766060 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4ebf5-f2ff-4f28-9733-3be3a36c9607 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:26:16.780253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4ebf5-f2ff-4f28-9733-3be3a36c9607 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:26:16.785783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c50995-a9db-45ea-b3b3-187a7b26f038 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:26:16.786367 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c50995-a9db-45ea-b3b3-187a7b26f038 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:16.786637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c50995-a9db-45ea-b3b3-187a7b26f038 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:16.786905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c50995-a9db-45ea-b3b3-187a7b26f038 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:16.794708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c50995-a9db-45ea-b3b3-187a7b26f038 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:26:16.796303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c50995-a9db-45ea-b3b3-187a7b26f038 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:26:16.805795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 775/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:26:20.832891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c00800-a0a6-4981-9011-68aeeb487f20 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:20.834010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c00800-a0a6-4981-9011-68aeeb487f20 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:20.834120 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c00800-a0a6-4981-9011-68aeeb487f20 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:20.835407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 753/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:26:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:26:20.840500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc975d-91f0-4ac2-bd6c-ceafe0da891d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:20.840933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc975d-91f0-4ac2-bd6c-ceafe0da891d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:20.841039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc975d-91f0-4ac2-bd6c-ceafe0da891d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:20.860111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-26cc975d-91f0-4ac2-bd6c-ceafe0da891d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:26:20.861293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-26cc975d-91f0-4ac2-bd6c-ceafe0da891d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:26:20.868926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 776/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:26:21.157542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c00800-a0a6-4981-9011-68aeeb487f20 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:26:21.159235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c4c00800-a0a6-4981-9011-68aeeb487f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:26:21.234222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c00800-a0a6-4981-9011-68aeeb487f20 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:26:21.237737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcc1b6-9fe8-43a9-ab0b-7faf7cc27459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:26:21.238326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcc1b6-9fe8-43a9-ab0b-7faf7cc27459 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:21.238604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcc1b6-9fe8-43a9-ab0b-7faf7cc27459 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:21.238877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcc1b6-9fe8-43a9-ab0b-7faf7cc27459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:21.248035 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dcc1b6-9fe8-43a9-ab0b-7faf7cc27459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:26:21.249462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dcc1b6-9fe8-43a9-ab0b-7faf7cc27459 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:26:21.259197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 754/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:26:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:26:23.406813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec46670-8cf7-4b31-8ae9-a8cb022e14d9 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:23.407353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec46670-8cf7-4b31-8ae9-a8cb022e14d9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:23.407460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec46670-8cf7-4b31-8ae9-a8cb022e14d9 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:23.408973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 777/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:26:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:26:23.414970 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0c736-591b-4729-8847-2cfa347889ca None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:23.415406 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0c736-591b-4729-8847-2cfa347889ca None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:23.415518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0c736-591b-4729-8847-2cfa347889ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:23.434572 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-68d0c736-591b-4729-8847-2cfa347889ca None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:26:23.435799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-68d0c736-591b-4729-8847-2cfa347889ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:26:23.443494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 755/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:26:23.727812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec46670-8cf7-4b31-8ae9-a8cb022e14d9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:26:23.738414 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec46670-8cf7-4b31-8ae9-a8cb022e14d9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:26:23.741963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84460c-942d-4fd0-8c97-fede8e2bf4f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:26:23.742552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84460c-942d-4fd0-8c97-fede8e2bf4f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:23.742833 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84460c-942d-4fd0-8c97-fede8e2bf4f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:23.743110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84460c-942d-4fd0-8c97-fede8e2bf4f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:23.751893 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84460c-942d-4fd0-8c97-fede8e2bf4f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:26:23.753318 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84460c-942d-4fd0-8c97-fede8e2bf4f7 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:26:23.763011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 778/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:26:27.327401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424d902-33ad-41c3-a697-cf17d604a4ee None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:27.327964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424d902-33ad-41c3-a697-cf17d604a4ee None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:27.327964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424d902-33ad-41c3-a697-cf17d604a4ee None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:27.329359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 756/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:26:27.335676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7a8ee-5402-47ca-944a-06468b6da1f1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:27.336104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7a8ee-5402-47ca-944a-06468b6da1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:27.336240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7a8ee-5402-47ca-944a-06468b6da1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:27.356199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d1f7a8ee-5402-47ca-944a-06468b6da1f1 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:26:27.357435 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d1f7a8ee-5402-47ca-944a-06468b6da1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:26:27.365321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 779/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:26:27.668754 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424d902-33ad-41c3-a697-cf17d604a4ee None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:26:27.670486 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5424d902-33ad-41c3-a697-cf17d604a4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:26:27.733997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:26:27.748024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424d902-33ad-41c3-a697-cf17d604a4ee None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:26:27.751613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a7ee9-f1db-4876-9c74-8e9a0cf14b97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:26:27.752213 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a7ee9-f1db-4876-9c74-8e9a0cf14b97 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:27.752495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a7ee9-f1db-4876-9c74-8e9a0cf14b97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:27.752772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a7ee9-f1db-4876-9c74-8e9a0cf14b97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:27.762138 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a7ee9-f1db-4876-9c74-8e9a0cf14b97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:26:27.763588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a7ee9-f1db-4876-9c74-8e9a0cf14b97 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:26:27.772830 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 757/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:26:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:26:29.932515 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47565a-c252-4b70-bbe1-968e10a98b92 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:29.932980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47565a-c252-4b70-bbe1-968e10a98b92 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:29.933098 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47565a-c252-4b70-bbe1-968e10a98b92 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:29.934397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 780/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:26:29.939719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690baa7-446e-42e6-858e-1f77a76edd0b None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:29.940130 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690baa7-446e-42e6-858e-1f77a76edd0b None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:29.940275 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690baa7-446e-42e6-858e-1f77a76edd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:29.960547 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-5690baa7-446e-42e6-858e-1f77a76edd0b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:26:29.961747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5690baa7-446e-42e6-858e-1f77a76edd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:26:29.969504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 758/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:26:30.283283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47565a-c252-4b70-bbe1-968e10a98b92 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:26:30.284984 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3a47565a-c252-4b70-bbe1-968e10a98b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:26:30.293232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=84 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:26:30.299741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47565a-c252-4b70-bbe1-968e10a98b92 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:26:30.303264 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5890fa9-7c76-4d8e-8734-9a56e62730a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:26:30.303902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5890fa9-7c76-4d8e-8734-9a56e62730a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:26:30.304187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5890fa9-7c76-4d8e-8734-9a56e62730a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:26:30.304462 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5890fa9-7c76-4d8e-8734-9a56e62730a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:26:30.313167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5890fa9-7c76-4d8e-8734-9a56e62730a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:26:30.314584 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5890fa9-7c76-4d8e-8734-9a56e62730a9 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:26:30.323845 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 781/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:26:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:26:39.955015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=11 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:26:40.319736 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:53.949361 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc9947-3b2b-42f2-a52e-53acd94c5502 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:53.951072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc9947-3b2b-42f2-a52e-53acd94c5502 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:53.951072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc9947-3b2b-42f2-a52e-53acd94c5502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:53.980284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-22dc9947-3b2b-42f2-a52e-53acd94c5502 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:53.981568 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-22dc9947-3b2b-42f2-a52e-53acd94c5502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:53.995562 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 759/1540] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:54.005924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ed5ce-3912-4155-aa6b-7f02f89c8a73 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.006543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ed5ce-3912-4155-aa6b-7f02f89c8a73 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.006543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ed5ce-3912-4155-aa6b-7f02f89c8a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.033188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-882ed5ce-3912-4155-aa6b-7f02f89c8a73 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.034492 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-882ed5ce-3912-4155-aa6b-7f02f89c8a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.046816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 782/1541] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:54.052656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc9947-3b2b-42f2-a52e-53acd94c5502 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.066182 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc9947-3b2b-42f2-a52e-53acd94c5502 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.073984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a30b8f-0092-461c-a0ce-a685e9667fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.074618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a30b8f-0092-461c-a0ce-a685e9667fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.074897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a30b8f-0092-461c-a0ce-a685e9667fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.075379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a30b8f-0092-461c-a0ce-a685e9667fa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.076188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a30b8f-0092-461c-a0ce-a685e9667fa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:27:54.076500 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a30b8f-0092-461c-a0ce-a685e9667fa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.078124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a30b8f-0092-461c-a0ce-a685e9667fa8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.096226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 760/1542] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:27:54.104564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ed5ce-3912-4155-aa6b-7f02f89c8a73 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.115046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ed5ce-3912-4155-aa6b-7f02f89c8a73 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.123834 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040edb2-1560-4575-8686-6690f6a4db5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.124446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040edb2-1560-4575-8686-6690f6a4db5b admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.124726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040edb2-1560-4575-8686-6690f6a4db5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.125001 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040edb2-1560-4575-8686-6690f6a4db5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.125648 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040edb2-1560-4575-8686-6690f6a4db5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.127585 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040edb2-1560-4575-8686-6690f6a4db5b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.180926 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2332f8-072a-43f6-8491-1ae20d693fed None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.181278 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2332f8-072a-43f6-8491-1ae20d693fed None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.181487 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2332f8-072a-43f6-8491-1ae20d693fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.186288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 783/1543] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/projects => generated 375 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:54.187613 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc5d23-d272-42c1-bf12-6acbc32346c5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.187910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc5d23-d272-42c1-bf12-6acbc32346c5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.188192 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc5d23-d272-42c1-bf12-6acbc32346c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.216643 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-7b2332f8-072a-43f6-8491-1ae20d693fed None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.216884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-febc5d23-d272-42c1-bf12-6acbc32346c5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.217871 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7b2332f8-072a-43f6-8491-1ae20d693fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.218002 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-febc5d23-d272-42c1-bf12-6acbc32346c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.339592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 761/1544] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:27:54.339811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 784/1545] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:27:54.339910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2332f8-072a-43f6-8491-1ae20d693fed None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.340013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a7a05-588a-4df3-99e9-05962743b7b9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.340013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a7a05-588a-4df3-99e9-05962743b7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.340176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a7a05-588a-4df3-99e9-05962743b7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.353139 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2332f8-072a-43f6-8491-1ae20d693fed None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.358907 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917622f6-9d0e-4f23-a3ee-6ed20665269e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.359528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917622f6-9d0e-4f23-a3ee-6ed20665269e admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.359852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917622f6-9d0e-4f23-a3ee-6ed20665269e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.360248 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917622f6-9d0e-4f23-a3ee-6ed20665269e admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.360889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917622f6-9d0e-4f23-a3ee-6ed20665269e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.363697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917622f6-9d0e-4f23-a3ee-6ed20665269e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.368092 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-711a7a05-588a-4df3-99e9-05962743b7b9 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.369312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-711a7a05-588a-4df3-99e9-05962743b7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.379736 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 785/1546] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:54.381196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8461418-d094-411a-8d07-e8f0003f7c4e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.381498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8461418-d094-411a-8d07-e8f0003f7c4e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.381766 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8461418-d094-411a-8d07-e8f0003f7c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.396511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 762/1547] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/users => generated 288 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:54.398338 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9919ee8-a9fc-4253-bffa-6b415a9cb5f0 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.398856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9919ee8-a9fc-4253-bffa-6b415a9cb5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.399475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9919ee8-a9fc-4253-bffa-6b415a9cb5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.404129 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a8461418-d094-411a-8d07-e8f0003f7c4e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.405336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a8461418-d094-411a-8d07-e8f0003f7c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.414373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 786/1548] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:54.415535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb33ef-0d92-48cf-a832-d20c41e59817 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.415856 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb33ef-0d92-48cf-a832-d20c41e59817 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.416151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb33ef-0d92-48cf-a832-d20c41e59817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.420673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-d9919ee8-a9fc-4253-bffa-6b415a9cb5f0 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.421911 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d9919ee8-a9fc-4253-bffa-6b415a9cb5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.430620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 763/1549] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:54.432099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b7d64-0113-4d59-8431-8e19da76e6fd None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.432327 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b7d64-0113-4d59-8431-8e19da76e6fd None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.432610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b7d64-0113-4d59-8431-8e19da76e6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.438425 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1dbb33ef-0d92-48cf-a832-d20c41e59817 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.439655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1dbb33ef-0d92-48cf-a832-d20c41e59817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.449210 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 787/1550] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:54.450398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c0c18-a777-4e24-8795-a85c118b4b5b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.450688 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c0c18-a777-4e24-8795-a85c118b4b5b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.451045 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c0c18-a777-4e24-8795-a85c118b4b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.452145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-d66b7d64-0113-4d59-8431-8e19da76e6fd None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.453544 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d66b7d64-0113-4d59-8431-8e19da76e6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.461349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 764/1551] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:54.462935 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805d6b5-b20e-464e-a18c-76fbd6cd1701 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.463226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805d6b5-b20e-464e-a18c-76fbd6cd1701 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.463499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805d6b5-b20e-464e-a18c-76fbd6cd1701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.470889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2a1c0c18-a777-4e24-8795-a85c118b4b5b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.472632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2a1c0c18-a777-4e24-8795-a85c118b4b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.481400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 788/1552] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:54.482556 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3988d7-584e-446d-8edc-b34cbca28460 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.482849 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3988d7-584e-446d-8edc-b34cbca28460 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.483126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3988d7-584e-446d-8edc-b34cbca28460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.483395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-6805d6b5-b20e-464e-a18c-76fbd6cd1701 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.484603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6805d6b5-b20e-464e-a18c-76fbd6cd1701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.492777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 765/1553] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:54.493621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805d6b5-b20e-464e-a18c-76fbd6cd1701 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.502463 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1b3988d7-584e-446d-8edc-b34cbca28460 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.503814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805d6b5-b20e-464e-a18c-76fbd6cd1701 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.504337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1b3988d7-584e-446d-8edc-b34cbca28460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.509008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce167bb-e468-4c2a-ae47-6c9cb878ffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.509590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce167bb-e468-4c2a-ae47-6c9cb878ffc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.509869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce167bb-e468-4c2a-ae47-6c9cb878ffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.510146 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce167bb-e468-4c2a-ae47-6c9cb878ffc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.510782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce167bb-e468-4c2a-ae47-6c9cb878ffc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.512511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce167bb-e468-4c2a-ae47-6c9cb878ffc8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.512886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 789/1554] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:54.514042 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ffdb7-d936-4923-b542-48e4cb7ce2e1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.514334 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ffdb7-d936-4923-b542-48e4cb7ce2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.514604 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ffdb7-d936-4923-b542-48e4cb7ce2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.523453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 766/1555] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:54.524774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244561b-f58f-4d94-af46-580db5106744 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.525125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244561b-f58f-4d94-af46-580db5106744 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.525404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244561b-f58f-4d94-af46-580db5106744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.535718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f31ffdb7-d936-4923-b542-48e4cb7ce2e1 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.536911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f31ffdb7-d936-4923-b542-48e4cb7ce2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.544765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 790/1556] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:54.545554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ffdb7-d936-4923-b542-48e4cb7ce2e1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.545729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-e244561b-f58f-4d94-af46-580db5106744 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.547049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e244561b-f58f-4d94-af46-580db5106744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.554732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ffdb7-d936-4923-b542-48e4cb7ce2e1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.559030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 767/1557] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:54.559030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244561b-f58f-4d94-af46-580db5106744 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.559583 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bd33a-2e35-41db-9e17-bdcebac762c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.560182 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bd33a-2e35-41db-9e17-bdcebac762c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.560456 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bd33a-2e35-41db-9e17-bdcebac762c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.560726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bd33a-2e35-41db-9e17-bdcebac762c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.561363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bd33a-2e35-41db-9e17-bdcebac762c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:27:54.561685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bd33a-2e35-41db-9e17-bdcebac762c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.562687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-7d9bd33a-2e35-41db-9e17-bdcebac762c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:27:54.562687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 12 23:27:54.562687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: Sep 12 23:27:54.563591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bd33a-2e35-41db-9e17-bdcebac762c9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.566308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244561b-f58f-4d94-af46-580db5106744 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.570901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5d248-1956-409c-8139-4699345b841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.571484 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5d248-1956-409c-8139-4699345b841d admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.571801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5d248-1956-409c-8139-4699345b841d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.572079 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5d248-1956-409c-8139-4699345b841d admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.572719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5d248-1956-409c-8139-4699345b841d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:27:54.573023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5d248-1956-409c-8139-4699345b841d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.574743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5d248-1956-409c-8139-4699345b841d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.575527 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 791/1558] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:27:54.576750 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb6b9-a97b-47dc-b85e-3a1fd5e7a2a1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.577039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb6b9-a97b-47dc-b85e-3a1fd5e7a2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.577312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb6b9-a97b-47dc-b85e-3a1fd5e7a2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.587870 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 768/1559] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:27:54.589769 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5d248-1956-409c-8139-4699345b841d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.604906 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-69cdb6b9-a97b-47dc-b85e-3a1fd5e7a2a1 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.606646 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-69cdb6b9-a97b-47dc-b85e-3a1fd5e7a2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.607998 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5d248-1956-409c-8139-4699345b841d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.613168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215e22d-3cf6-49c9-b5f9-95159895ea9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.613796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215e22d-3cf6-49c9-b5f9-95159895ea9f admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.614077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215e22d-3cf6-49c9-b5f9-95159895ea9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.614381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215e22d-3cf6-49c9-b5f9-95159895ea9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.615025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215e22d-3cf6-49c9-b5f9-95159895ea9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:27:54.615332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215e22d-3cf6-49c9-b5f9-95159895ea9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.616641 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 792/1560] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:54.621226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215e22d-3cf6-49c9-b5f9-95159895ea9f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.621411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c64ae-ff20-40b5-8d1b-33a0c505235d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.621411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c64ae-ff20-40b5-8d1b-33a0c505235d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.621411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c64ae-ff20-40b5-8d1b-33a0c505235d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.629582 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 769/1561] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:27:54.630553 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215e22d-3cf6-49c9-b5f9-95159895ea9f admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.637611 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-072c64ae-ff20-40b5-8d1b-33a0c505235d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:54.639035 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-072c64ae-ff20-40b5-8d1b-33a0c505235d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:54.641209 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215e22d-3cf6-49c9-b5f9-95159895ea9f admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.645973 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099bd59-1cb5-4314-aff6-b430402edb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.646563 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099bd59-1cb5-4314-aff6-b430402edb00 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.646839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099bd59-1cb5-4314-aff6-b430402edb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.646959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 793/1562] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:54.647109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099bd59-1cb5-4314-aff6-b430402edb00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.647772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c64ae-ff20-40b5-8d1b-33a0c505235d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.647891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099bd59-1cb5-4314-aff6-b430402edb00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.649469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099bd59-1cb5-4314-aff6-b430402edb00 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.657316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c64ae-ff20-40b5-8d1b-33a0c505235d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.660251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 770/1563] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:54.661033 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099bd59-1cb5-4314-aff6-b430402edb00 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.662651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c28375-84c0-4a3d-b481-a01d8feeb240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.663327 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c28375-84c0-4a3d-b481-a01d8feeb240 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.663511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c28375-84c0-4a3d-b481-a01d8feeb240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.663811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c28375-84c0-4a3d-b481-a01d8feeb240 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.664473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c28375-84c0-4a3d-b481-a01d8feeb240 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:27:54.664776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c28375-84c0-4a3d-b481-a01d8feeb240 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.666350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c28375-84c0-4a3d-b481-a01d8feeb240 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.670043 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099bd59-1cb5-4314-aff6-b430402edb00 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.675133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750d1fb-b1bd-456a-9a91-95bf993e5636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.675774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750d1fb-b1bd-456a-9a91-95bf993e5636 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.676051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750d1fb-b1bd-456a-9a91-95bf993e5636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.676322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750d1fb-b1bd-456a-9a91-95bf993e5636 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.676942 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750d1fb-b1bd-456a-9a91-95bf993e5636 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:27:54.677243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750d1fb-b1bd-456a-9a91-95bf993e5636 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.685471 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 794/1564] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:27:54.685646 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750d1fb-b1bd-456a-9a91-95bf993e5636 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.685759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c28375-84c0-4a3d-b481-a01d8feeb240 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.688954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c28375-84c0-4a3d-b481-a01d8feeb240 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.692034 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 771/1565] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:27:54.692682 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750d1fb-b1bd-456a-9a91-95bf993e5636 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.700332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28246ef-abb5-4991-829c-6ed4454d3d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.700928 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28246ef-abb5-4991-829c-6ed4454d3d8f admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.701194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28246ef-abb5-4991-829c-6ed4454d3d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.701467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28246ef-abb5-4991-829c-6ed4454d3d8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.702082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28246ef-abb5-4991-829c-6ed4454d3d8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.702191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750d1fb-b1bd-456a-9a91-95bf993e5636 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.704010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28246ef-abb5-4991-829c-6ed4454d3d8f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.707301 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18147666-194f-4993-b730-f6ba6f9758d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.707906 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18147666-194f-4993-b730-f6ba6f9758d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.708191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18147666-194f-4993-b730-f6ba6f9758d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.708467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18147666-194f-4993-b730-f6ba6f9758d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.709087 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18147666-194f-4993-b730-f6ba6f9758d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.710167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: WARNING py.warnings [None req-18147666-194f-4993-b730-f6ba6f9758d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:27:54.710167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: warnings.warn(msg) Sep 12 23:27:54.710167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: Sep 12 23:27:54.711350 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18147666-194f-4993-b730-f6ba6f9758d7 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.735080 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 795/1566] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/projects => generated 393 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:54.735720 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28246ef-abb5-4991-829c-6ed4454d3d8f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.742593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 772/1567] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:54.743391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18147666-194f-4993-b730-f6ba6f9758d7 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.759924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:54.760862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:54.800616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28246ef-abb5-4991-829c-6ed4454d3d8f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.810958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18147666-194f-4993-b730-f6ba6f9758d7 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.815782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720f544-19f2-4235-850f-df0717198c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.816386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f544-19f2-4235-850f-df0717198c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.816692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f544-19f2-4235-850f-df0717198c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.816969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f544-19f2-4235-850f-df0717198c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.817597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720f544-19f2-4235-850f-df0717198c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.819515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720f544-19f2-4235-850f-df0717198c75 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.847884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 773/1568] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:54.848641 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720f544-19f2-4235-850f-df0717198c75 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.861880 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b55d2-745f-4509-9246-37178fb37b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.862560 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b55d2-745f-4509-9246-37178fb37b17 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.862848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b55d2-745f-4509-9246-37178fb37b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.863123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b55d2-745f-4509-9246-37178fb37b17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.863799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b55d2-745f-4509-9246-37178fb37b17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:27:54.864107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b55d2-745f-4509-9246-37178fb37b17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.865712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b55d2-745f-4509-9246-37178fb37b17 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.887154 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 796/1569] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:27:54.887154 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b55d2-745f-4509-9246-37178fb37b17 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.891347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b55d2-745f-4509-9246-37178fb37b17 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.919091 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720f544-19f2-4235-850f-df0717198c75 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.924795 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6e6d7-ee64-4aa2-bd22-2305a52d5617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.925534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6e6d7-ee64-4aa2-bd22-2305a52d5617 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.925920 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6e6d7-ee64-4aa2-bd22-2305a52d5617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.926295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6e6d7-ee64-4aa2-bd22-2305a52d5617 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.927025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6e6d7-ee64-4aa2-bd22-2305a52d5617 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:27:54.927366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6e6d7-ee64-4aa2-bd22-2305a52d5617 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.929789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6e6d7-ee64-4aa2-bd22-2305a52d5617 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.941701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 774/1570] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:27:54.942533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6e6d7-ee64-4aa2-bd22-2305a52d5617 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.954022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6e6d7-ee64-4aa2-bd22-2305a52d5617 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:54.959322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebca286-2431-4a8a-97cc-4b5b4d92e6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.960061 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebca286-2431-4a8a-97cc-4b5b4d92e6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.960399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebca286-2431-4a8a-97cc-4b5b4d92e6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.960674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebca286-2431-4a8a-97cc-4b5b4d92e6b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.961114 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a771125-bb24-4142-9894-50f842a301d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:54.961299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebca286-2431-4a8a-97cc-4b5b4d92e6b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.961783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a771125-bb24-4142-9894-50f842a301d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:54.962064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a771125-bb24-4142-9894-50f842a301d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:54.962347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a771125-bb24-4142-9894-50f842a301d7 admin admin] PATH_INFO: `/v3/projects/b8ffe803ce4b4d0181be62e1dfb094bf/users/7f56dde21c754cfe8253ec5c1b7b90d7/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:54.963324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebca286-2431-4a8a-97cc-4b5b4d92e6b0 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.978403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a771125-bb24-4142-9894-50f842a301d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ffe803ce4b4d0181be62e1dfb094bf, user_id=7f56dde21c754cfe8253ec5c1b7b90d7, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:54.981252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a771125-bb24-4142-9894-50f842a301d7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:54.994867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 775/1571] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:54.994867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebca286-2431-4a8a-97cc-4b5b4d92e6b0 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:54.997690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 797/1572] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:27:54 2022] PUT /identity/v3/projects/b8ffe803ce4b4d0181be62e1dfb094bf/users/7f56dde21c754cfe8253ec5c1b7b90d7/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:27:54.998556 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a771125-bb24-4142-9894-50f842a301d7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.048662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:55.055707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebca286-2431-4a8a-97cc-4b5b4d92e6b0 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.060622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2795a-d814-449d-9c46-b896b5bb1365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.061442 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2795a-d814-449d-9c46-b896b5bb1365 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.061799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2795a-d814-449d-9c46-b896b5bb1365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.062145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2795a-d814-449d-9c46-b896b5bb1365 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.062846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2795a-d814-449d-9c46-b896b5bb1365 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.063803 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a771125-bb24-4142-9894-50f842a301d7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.064871 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2795a-d814-449d-9c46-b896b5bb1365 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.068951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538d077-66f4-43f3-bc15-eaffff2d2a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.069550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538d077-66f4-43f3-bc15-eaffff2d2a1a admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.070008 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538d077-66f4-43f3-bc15-eaffff2d2a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.070286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538d077-66f4-43f3-bc15-eaffff2d2a1a admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.071246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538d077-66f4-43f3-bc15-eaffff2d2a1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.072299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-d538d077-66f4-43f3-bc15-eaffff2d2a1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:27:55.072299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 12 23:27:55.072299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: Sep 12 23:27:55.073495 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538d077-66f4-43f3-bc15-eaffff2d2a1a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.081285 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:55.094897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 776/1573] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:55.095810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2795a-d814-449d-9c46-b896b5bb1365 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.099705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 798/1574] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:27:54 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:55.099705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538d077-66f4-43f3-bc15-eaffff2d2a1a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.107239 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538d077-66f4-43f3-bc15-eaffff2d2a1a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.160704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2795a-d814-449d-9c46-b896b5bb1365 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.163370 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6135df-b8e1-4ef4-b042-e82162d1cc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.164094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6135df-b8e1-4ef4-b042-e82162d1cc7f admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.164376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6135df-b8e1-4ef4-b042-e82162d1cc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.164652 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6135df-b8e1-4ef4-b042-e82162d1cc7f admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.165311 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6135df-b8e1-4ef4-b042-e82162d1cc7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.165416 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c6ddb-e765-4554-acea-5ebe4de8f432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.166019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c6ddb-e765-4554-acea-5ebe4de8f432 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.166300 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c6ddb-e765-4554-acea-5ebe4de8f432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.166576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c6ddb-e765-4554-acea-5ebe4de8f432 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.167199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c6ddb-e765-4554-acea-5ebe4de8f432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.167306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6135df-b8e1-4ef4-b042-e82162d1cc7f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.169141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c6ddb-e765-4554-acea-5ebe4de8f432 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.189943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 799/1575] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:27:55 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:55.191310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 777/1576] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:27:55 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:55.191804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6135df-b8e1-4ef4-b042-e82162d1cc7f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.192145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c6ddb-e765-4554-acea-5ebe4de8f432 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.202215 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6135df-b8e1-4ef4-b042-e82162d1cc7f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.203094 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c6ddb-e765-4554-acea-5ebe4de8f432 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.207346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a214e-9506-4c9b-8655-651849b71ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.207982 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a214e-9506-4c9b-8655-651849b71ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.208263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a214e-9506-4c9b-8655-651849b71ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.208367 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad81873-62cb-4270-b4cd-94cab68d397d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.208540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a214e-9506-4c9b-8655-651849b71ca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.208891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad81873-62cb-4270-b4cd-94cab68d397d admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.209174 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad81873-62cb-4270-b4cd-94cab68d397d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.209279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a214e-9506-4c9b-8655-651849b71ca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.209450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad81873-62cb-4270-b4cd-94cab68d397d admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.210104 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad81873-62cb-4270-b4cd-94cab68d397d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.211208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a214e-9506-4c9b-8655-651849b71ca0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.212066 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad81873-62cb-4270-b4cd-94cab68d397d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.239154 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 800/1577] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:27:55 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:55.239997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a214e-9506-4c9b-8655-651849b71ca0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.243824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 778/1578] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:27:55 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:55.243824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9583ed3-b57f-4d04-99d0-b9adfcb49bf5 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.243824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9583ed3-b57f-4d04-99d0-b9adfcb49bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.243824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9583ed3-b57f-4d04-99d0-b9adfcb49bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.261105 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c9583ed3-b57f-4d04-99d0-b9adfcb49bf5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:55.262325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c9583ed3-b57f-4d04-99d0-b9adfcb49bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:55.307528 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a214e-9506-4c9b-8655-651849b71ca0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.326048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 779/1579] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:55.326957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9583ed3-b57f-4d04-99d0-b9adfcb49bf5 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.337017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9583ed3-b57f-4d04-99d0-b9adfcb49bf5 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.341623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ecfb3-4c67-459b-97ee-130ba4dc853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.342222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ecfb3-4c67-459b-97ee-130ba4dc853a admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.342506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ecfb3-4c67-459b-97ee-130ba4dc853a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.342807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ecfb3-4c67-459b-97ee-130ba4dc853a admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.343437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ecfb3-4c67-459b-97ee-130ba4dc853a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.345387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ecfb3-4c67-459b-97ee-130ba4dc853a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.365386 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:55.370847 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 780/1580] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:27:55 2022] POST /identity/v3/users => generated 294 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:55.370847 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ecfb3-4c67-459b-97ee-130ba4dc853a admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.376375 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903894d-8b7f-44bf-9d08-3aeae79aa375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.377191 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903894d-8b7f-44bf-9d08-3aeae79aa375 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.378283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903894d-8b7f-44bf-9d08-3aeae79aa375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.378631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903894d-8b7f-44bf-9d08-3aeae79aa375 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.378732 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ecfb3-4c67-459b-97ee-130ba4dc853a admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.379391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903894d-8b7f-44bf-9d08-3aeae79aa375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.381457 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903894d-8b7f-44bf-9d08-3aeae79aa375 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.383419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67604533-3a79-4c87-90e8-9b54cd8fa911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.384036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67604533-3a79-4c87-90e8-9b54cd8fa911 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.384312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67604533-3a79-4c87-90e8-9b54cd8fa911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.384581 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67604533-3a79-4c87-90e8-9b54cd8fa911 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.385213 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67604533-3a79-4c87-90e8-9b54cd8fa911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.386846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67604533-3a79-4c87-90e8-9b54cd8fa911 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.398841 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 781/1581] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:55.399753 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67604533-3a79-4c87-90e8-9b54cd8fa911 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.405071 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 801/1582] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:27:55 2022] POST /identity/v3/users => generated 303 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:55.406563 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903894d-8b7f-44bf-9d08-3aeae79aa375 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.411887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67604533-3a79-4c87-90e8-9b54cd8fa911 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.416229 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903894d-8b7f-44bf-9d08-3aeae79aa375 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.417433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2311421-61bc-42c0-bb97-118aee3bfcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.418158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2311421-61bc-42c0-bb97-118aee3bfcf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.418437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2311421-61bc-42c0-bb97-118aee3bfcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.418711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2311421-61bc-42c0-bb97-118aee3bfcf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.419349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2311421-61bc-42c0-bb97-118aee3bfcf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.421015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2311421-61bc-42c0-bb97-118aee3bfcf4 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.421146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ca032-1b4c-40f0-845e-159da6a4d2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.421763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ca032-1b4c-40f0-845e-159da6a4d2ef admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.422107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ca032-1b4c-40f0-845e-159da6a4d2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.422380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ca032-1b4c-40f0-845e-159da6a4d2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.423017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ca032-1b4c-40f0-845e-159da6a4d2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.424642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ca032-1b4c-40f0-845e-159da6a4d2ef admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.434671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 782/1583] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:55.434671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2311421-61bc-42c0-bb97-118aee3bfcf4 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.441483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 802/1584] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:55.441483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ca032-1b4c-40f0-845e-159da6a4d2ef admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.444516 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2311421-61bc-42c0-bb97-118aee3bfcf4 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.449378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ca032-1b4c-40f0-845e-159da6a4d2ef admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.452092 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec436f3-d76d-4f82-9a33-80394efd30d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.452680 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec436f3-d76d-4f82-9a33-80394efd30d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.452962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec436f3-d76d-4f82-9a33-80394efd30d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.453269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec436f3-d76d-4f82-9a33-80394efd30d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.453897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec436f3-d76d-4f82-9a33-80394efd30d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.454021 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02a0d1-b6f7-4bcd-bcfb-e57143258b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.454539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02a0d1-b6f7-4bcd-bcfb-e57143258b89 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.454817 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02a0d1-b6f7-4bcd-bcfb-e57143258b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.455087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02a0d1-b6f7-4bcd-bcfb-e57143258b89 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.456337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02a0d1-b6f7-4bcd-bcfb-e57143258b89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.456447 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec436f3-d76d-4f82-9a33-80394efd30d9 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.458195 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02a0d1-b6f7-4bcd-bcfb-e57143258b89 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.478630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 783/1585] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:27:55 2022] POST /identity/v3/users => generated 301 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:55.479296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 803/1586] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:27:55 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:55.480011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02a0d1-b6f7-4bcd-bcfb-e57143258b89 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.482615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec436f3-d76d-4f82-9a33-80394efd30d9 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.489954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02a0d1-b6f7-4bcd-bcfb-e57143258b89 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.491465 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:55.494869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5238e-6736-42fd-9e3e-9f019e97658f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.495480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5238e-6736-42fd-9e3e-9f019e97658f admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.495790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5238e-6736-42fd-9e3e-9f019e97658f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.496057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5238e-6736-42fd-9e3e-9f019e97658f admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.496693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5238e-6736-42fd-9e3e-9f019e97658f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.498337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5238e-6736-42fd-9e3e-9f019e97658f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.508138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 804/1587] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:55.508138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5238e-6736-42fd-9e3e-9f019e97658f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.518787 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5238e-6736-42fd-9e3e-9f019e97658f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.523513 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125d419-c1e9-454f-b25c-a6a36cc95498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.524173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125d419-c1e9-454f-b25c-a6a36cc95498 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.524460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125d419-c1e9-454f-b25c-a6a36cc95498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.524778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125d419-c1e9-454f-b25c-a6a36cc95498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.525430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125d419-c1e9-454f-b25c-a6a36cc95498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.527166 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125d419-c1e9-454f-b25c-a6a36cc95498 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.541072 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 805/1588] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:55.541072 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125d419-c1e9-454f-b25c-a6a36cc95498 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.543608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec436f3-d76d-4f82-9a33-80394efd30d9 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.547316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3671ccb-59da-4dd9-a6db-08856fcc3256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.547972 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3671ccb-59da-4dd9-a6db-08856fcc3256 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.548251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3671ccb-59da-4dd9-a6db-08856fcc3256 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.548522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3671ccb-59da-4dd9-a6db-08856fcc3256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.548688 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125d419-c1e9-454f-b25c-a6a36cc95498 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.553240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ad24b-48bf-4b3e-a9c2-713c8bfd96ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.553802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ad24b-48bf-4b3e-a9c2-713c8bfd96ed admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.554193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ad24b-48bf-4b3e-a9c2-713c8bfd96ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.554353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ad24b-48bf-4b3e-a9c2-713c8bfd96ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.554972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ad24b-48bf-4b3e-a9c2-713c8bfd96ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.556545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ad24b-48bf-4b3e-a9c2-713c8bfd96ed admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.557550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3671ccb-59da-4dd9-a6db-08856fcc3256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.558965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3671ccb-59da-4dd9-a6db-08856fcc3256 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.566817 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 806/1589] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:55.568328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 784/1590] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:27:55.568953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ad24b-48bf-4b3e-a9c2-713c8bfd96ed admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.569074 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3671ccb-59da-4dd9-a6db-08856fcc3256 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.577859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3671ccb-59da-4dd9-a6db-08856fcc3256 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.580472 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ad24b-48bf-4b3e-a9c2-713c8bfd96ed admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.583101 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7598b8-b67b-479f-a27f-6368b11161ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.583716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7598b8-b67b-479f-a27f-6368b11161ce admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.583996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7598b8-b67b-479f-a27f-6368b11161ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.584272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7598b8-b67b-479f-a27f-6368b11161ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.584900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7598b8-b67b-479f-a27f-6368b11161ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.586500 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7598b8-b67b-479f-a27f-6368b11161ce admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.586634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30392f-72f5-47ec-a670-69731a6ec12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.588055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30392f-72f5-47ec-a670-69731a6ec12a admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.588055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30392f-72f5-47ec-a670-69731a6ec12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.588055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30392f-72f5-47ec-a670-69731a6ec12a admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.588833 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30392f-72f5-47ec-a670-69731a6ec12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.590502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30392f-72f5-47ec-a670-69731a6ec12a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.598508 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 785/1591] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:55.599280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7598b8-b67b-479f-a27f-6368b11161ce admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.605234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 807/1592] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:55.607086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30392f-72f5-47ec-a670-69731a6ec12a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.609528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7598b8-b67b-479f-a27f-6368b11161ce admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.614752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec725032-915b-4fef-aa3c-fc6bd5bbb968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.615796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec725032-915b-4fef-aa3c-fc6bd5bbb968 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.616158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec725032-915b-4fef-aa3c-fc6bd5bbb968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.616538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec725032-915b-4fef-aa3c-fc6bd5bbb968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.617322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec725032-915b-4fef-aa3c-fc6bd5bbb968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.617508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30392f-72f5-47ec-a670-69731a6ec12a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.619105 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec725032-915b-4fef-aa3c-fc6bd5bbb968 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.622795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db08791-1342-459a-be51-f75aa43e5bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.623394 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db08791-1342-459a-be51-f75aa43e5bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.623708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db08791-1342-459a-be51-f75aa43e5bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.624278 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db08791-1342-459a-be51-f75aa43e5bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.624618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db08791-1342-459a-be51-f75aa43e5bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.626657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db08791-1342-459a-be51-f75aa43e5bf4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.634505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 786/1593] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:55.634505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec725032-915b-4fef-aa3c-fc6bd5bbb968 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.639662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 808/1594] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:55.639662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db08791-1342-459a-be51-f75aa43e5bf4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.645195 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec725032-915b-4fef-aa3c-fc6bd5bbb968 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.650501 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b1284-760e-4ad5-86f5-1416b789fd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.651028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db08791-1342-459a-be51-f75aa43e5bf4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.651492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b1284-760e-4ad5-86f5-1416b789fd29 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.651837 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b1284-760e-4ad5-86f5-1416b789fd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.652413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b1284-760e-4ad5-86f5-1416b789fd29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.652865 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b1284-760e-4ad5-86f5-1416b789fd29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.654489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b1284-760e-4ad5-86f5-1416b789fd29 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.656817 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bcf57-ef37-4e85-93f2-7c9cee900e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.657558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bcf57-ef37-4e85-93f2-7c9cee900e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.657776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bcf57-ef37-4e85-93f2-7c9cee900e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.658018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bcf57-ef37-4e85-93f2-7c9cee900e2e admin admin] PATH_INFO: `/v3/projects/adc049ff85c14935a9f68be76cd61c05/users/0248519bf5b54ce3ba3b40f2c9a4c628/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.668526 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 787/1595] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:55.669371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b1284-760e-4ad5-86f5-1416b789fd29 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.678853 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bcf57-ef37-4e85-93f2-7c9cee900e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=adc049ff85c14935a9f68be76cd61c05, user_id=0248519bf5b54ce3ba3b40f2c9a4c628, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.680547 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b1284-760e-4ad5-86f5-1416b789fd29 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.682122 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bcf57-ef37-4e85-93f2-7c9cee900e2e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.686098 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c25cff-18aa-4368-84b3-a078829873bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.686755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c25cff-18aa-4368-84b3-a078829873bd admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.687005 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c25cff-18aa-4368-84b3-a078829873bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.687315 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c25cff-18aa-4368-84b3-a078829873bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.688069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c25cff-18aa-4368-84b3-a078829873bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.689732 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c25cff-18aa-4368-84b3-a078829873bd admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.696881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 809/1596] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:27:55 2022] PUT /identity/v3/projects/adc049ff85c14935a9f68be76cd61c05/users/0248519bf5b54ce3ba3b40f2c9a4c628/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:27:55.700583 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bcf57-ef37-4e85-93f2-7c9cee900e2e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.700986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 788/1597] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:55.701956 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c25cff-18aa-4368-84b3-a078829873bd admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.712368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c25cff-18aa-4368-84b3-a078829873bd admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.775498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bcf57-ef37-4e85-93f2-7c9cee900e2e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.776548 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ec7d1-387a-4b2f-bd0e-ab1c57f27faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.777320 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ec7d1-387a-4b2f-bd0e-ab1c57f27faa admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.777663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ec7d1-387a-4b2f-bd0e-ab1c57f27faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.777950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ec7d1-387a-4b2f-bd0e-ab1c57f27faa admin admin] PATH_INFO: `/v3/projects/c2bcb31c2c0646ff80d50c51c842db89/users/32e5b36aa97747d798b79509f3f66dd5/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.781109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455d02-9a08-4c58-9db7-b45fcc7bfa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.781666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455d02-9a08-4c58-9db7-b45fcc7bfa1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.781947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455d02-9a08-4c58-9db7-b45fcc7bfa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.782248 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455d02-9a08-4c58-9db7-b45fcc7bfa1c admin admin] PATH_INFO: `/v3/projects/1b1b14efb0dd42fcad321d9b5c8dafed/users/582ac771652d49f7a79facf89374e5bf/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.802878 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87455d02-9a08-4c58-9db7-b45fcc7bfa1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b1b14efb0dd42fcad321d9b5c8dafed, user_id=582ac771652d49f7a79facf89374e5bf, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.804606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ec7d1-387a-4b2f-bd0e-ab1c57f27faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2bcb31c2c0646ff80d50c51c842db89, user_id=32e5b36aa97747d798b79509f3f66dd5, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.805712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87455d02-9a08-4c58-9db7-b45fcc7bfa1c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.806683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: WARNING py.warnings [None req-635ec7d1-387a-4b2f-bd0e-ab1c57f27faa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:27:55.806683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: warnings.warn(msg) Sep 12 23:27:55.806683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: Sep 12 23:27:55.809375 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ec7d1-387a-4b2f-bd0e-ab1c57f27faa admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.821123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 810/1598] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:27:55 2022] PUT /identity/v3/projects/1b1b14efb0dd42fcad321d9b5c8dafed/users/582ac771652d49f7a79facf89374e5bf/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:27:55.821956 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455d02-9a08-4c58-9db7-b45fcc7bfa1c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.836875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 789/1599] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:27:55 2022] PUT /identity/v3/projects/c2bcb31c2c0646ff80d50c51c842db89/users/32e5b36aa97747d798b79509f3f66dd5/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:27:55.838279 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ec7d1-387a-4b2f-bd0e-ab1c57f27faa admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.849831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:55.898501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455d02-9a08-4c58-9db7-b45fcc7bfa1c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.903495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:55.927713 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ec7d1-387a-4b2f-bd0e-ab1c57f27faa admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.933959 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cbb20-5487-45c7-a61f-ea4ac618a64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.934891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cbb20-5487-45c7-a61f-ea4ac618a64a admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.935285 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cbb20-5487-45c7-a61f-ea4ac618a64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.935714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cbb20-5487-45c7-a61f-ea4ac618a64a admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.936469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15cbb20-5487-45c7-a61f-ea4ac618a64a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.938393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15cbb20-5487-45c7-a61f-ea4ac618a64a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.953834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 790/1600] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:55.954867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cbb20-5487-45c7-a61f-ea4ac618a64a admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:55.965663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cbb20-5487-45c7-a61f-ea4ac618a64a admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:55.972073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd79e3-8dd4-43e7-b2ba-24f712e80fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.973018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd79e3-8dd4-43e7-b2ba-24f712e80fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.973544 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd79e3-8dd4-43e7-b2ba-24f712e80fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.974058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd79e3-8dd4-43e7-b2ba-24f712e80fc9 admin admin] PATH_INFO: `/v3/projects/5aa1c574bdd14a1bb0ae5f82f892e6bf/users/2749a61b3dd04d89a5c2e4235aa96934/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:55.990833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd79e3-8dd4-43e7-b2ba-24f712e80fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aa1c574bdd14a1bb0ae5f82f892e6bf, user_id=2749a61b3dd04d89a5c2e4235aa96934, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:55.993898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd79e3-8dd4-43e7-b2ba-24f712e80fc9 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:55.996340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9281c-aad0-401d-8bbc-c5093e06520c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:55.997459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9281c-aad0-401d-8bbc-c5093e06520c admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:55.998633 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9281c-aad0-401d-8bbc-c5093e06520c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:55.999299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9281c-aad0-401d-8bbc-c5093e06520c admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.000137 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9281c-aad0-401d-8bbc-c5093e06520c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:56.002637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9281c-aad0-401d-8bbc-c5093e06520c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:56.012588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 791/1601] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:27:55 2022] PUT /identity/v3/projects/5aa1c574bdd14a1bb0ae5f82f892e6bf/users/2749a61b3dd04d89a5c2e4235aa96934/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:27:56.014873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2561a67-38ee-4bcc-b3a4-f69604cd3aa8 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.018339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 811/1602] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:56.018339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9281c-aad0-401d-8bbc-c5093e06520c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:56.020337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2561a67-38ee-4bcc-b3a4-f69604cd3aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.020337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2561a67-38ee-4bcc-b3a4-f69604cd3aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.027381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9281c-aad0-401d-8bbc-c5093e06520c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:56.040876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-f2561a67-38ee-4bcc-b3a4-f69604cd3aa8 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:56.042119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-f2561a67-38ee-4bcc-b3a4-f69604cd3aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:56.100259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b29265-ad78-4e52-966f-5f072af8bd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:56.101185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b29265-ad78-4e52-966f-5f072af8bd0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.101497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b29265-ad78-4e52-966f-5f072af8bd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.101782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b29265-ad78-4e52-966f-5f072af8bd0b admin admin] PATH_INFO: `/v3/projects/45befad61bc94b1aafb57766cfad5be4/users/17715e1c91a246489280bf04ed7fae0b/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.109744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 792/1603] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:56.111083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a487380-28eb-43c7-9cbe-7c6f467fbc7c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.111370 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a487380-28eb-43c7-9cbe-7c6f467fbc7c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.111707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a487380-28eb-43c7-9cbe-7c6f467fbc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.122295 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b29265-ad78-4e52-966f-5f072af8bd0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=45befad61bc94b1aafb57766cfad5be4, user_id=17715e1c91a246489280bf04ed7fae0b, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:56.125678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b29265-ad78-4e52-966f-5f072af8bd0b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:56.140890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 812/1604] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:27:56 2022] PUT /identity/v3/projects/45befad61bc94b1aafb57766cfad5be4/users/17715e1c91a246489280bf04ed7fae0b/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:27:56.142185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-8a487380-28eb-43c7-9cbe-7c6f467fbc7c None None] MFA Rules not processed for user `582ac771652d49f7a79facf89374e5bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:56.142352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8b112-92d1-41be-acd8-e5013f169f8f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.142635 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8b112-92d1-41be-acd8-e5013f169f8f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.142914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8b112-92d1-41be-acd8-e5013f169f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.144239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8a487380-28eb-43c7-9cbe-7c6f467fbc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:56.171412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7bd8b112-92d1-41be-acd8-e5013f169f8f None None] MFA Rules not processed for user `32e5b36aa97747d798b79509f3f66dd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:56.172708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7bd8b112-92d1-41be-acd8-e5013f169f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:56.206259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:56.213759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 793/1605] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:56.214877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a487380-28eb-43c7-9cbe-7c6f467fbc7c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:56.267847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 813/1606] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:56.269895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9d3ef-e043-4f0d-8a3e-2ca4f939292f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.270161 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9d3ef-e043-4f0d-8a3e-2ca4f939292f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.270442 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9d3ef-e043-4f0d-8a3e-2ca4f939292f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.287835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:56.291347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-aca9d3ef-e043-4f0d-8a3e-2ca4f939292f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:56.292827 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-aca9d3ef-e043-4f0d-8a3e-2ca4f939292f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:56.311257 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a487380-28eb-43c7-9cbe-7c6f467fbc7c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:56.316495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3cc9b-c4c2-48ea-8ebf-3ec23fb639f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:56.317261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3cc9b-c4c2-48ea-8ebf-3ec23fb639f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.317594 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3cc9b-c4c2-48ea-8ebf-3ec23fb639f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.317937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3cc9b-c4c2-48ea-8ebf-3ec23fb639f0 admin admin] PATH_INFO: `/v3/projects/811d8512813f49d79facec817fc5d03e/users/98da6780db2d43d5be8e971dd2d86119/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.335926 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3cc9b-c4c2-48ea-8ebf-3ec23fb639f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=811d8512813f49d79facec817fc5d03e, user_id=98da6780db2d43d5be8e971dd2d86119, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:56.339706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3cc9b-c4c2-48ea-8ebf-3ec23fb639f0 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:56.352849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 794/1607] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:27:56 2022] PUT /identity/v3/projects/811d8512813f49d79facec817fc5d03e/users/98da6780db2d43d5be8e971dd2d86119/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:27:56.353657 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3cc9b-c4c2-48ea-8ebf-3ec23fb639f0 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:56.361503 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 814/1608] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:56.361503 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9d3ef-e043-4f0d-8a3e-2ca4f939292f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:56.447593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9d3ef-e043-4f0d-8a3e-2ca4f939292f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:56.450694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3cc9b-c4c2-48ea-8ebf-3ec23fb639f0 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:56.465652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf45f1-a635-4508-9345-45162fad0f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:56.466512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf45f1-a635-4508-9345-45162fad0f88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.466930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf45f1-a635-4508-9345-45162fad0f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.467420 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf45f1-a635-4508-9345-45162fad0f88 admin admin] PATH_INFO: `/v3/projects/3055070ceb48447e9c6e05d102ad748f/users/a29f8dda51c44e08a46fabe3fe9f86ce/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.487988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf45f1-a635-4508-9345-45162fad0f88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3055070ceb48447e9c6e05d102ad748f, user_id=a29f8dda51c44e08a46fabe3fe9f86ce, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:56.491284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf45f1-a635-4508-9345-45162fad0f88 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:56.508770 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f746b-bdcf-46dd-a67c-a8ad621493a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:56.509518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f746b-bdcf-46dd-a67c-a8ad621493a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.509709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f746b-bdcf-46dd-a67c-a8ad621493a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.510033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f746b-bdcf-46dd-a67c-a8ad621493a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.511849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 795/1609] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:27:56 2022] PUT /identity/v3/projects/3055070ceb48447e9c6e05d102ad748f/users/a29f8dda51c44e08a46fabe3fe9f86ce/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:27:56.514201 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4325ba-26b0-4918-9440-2962bc45a4a3 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.514495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4325ba-26b0-4918-9440-2962bc45a4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.514940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4325ba-26b0-4918-9440-2962bc45a4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.526997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f746b-bdcf-46dd-a67c-a8ad621493a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:56.529185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f746b-bdcf-46dd-a67c-a8ad621493a2 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:56.551872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 815/1610] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:27:56.553531 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896d157-f55b-43de-b2af-73f3fdc0d6b3 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.555937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896d157-f55b-43de-b2af-73f3fdc0d6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.556967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896d157-f55b-43de-b2af-73f3fdc0d6b3 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.559956 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-be4325ba-26b0-4918-9440-2962bc45a4a3 None None] MFA Rules not processed for user `17715e1c91a246489280bf04ed7fae0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:56.560407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 816/1611] 10.222.0.27 () {54 vars in 962 bytes} [Mon Sep 12 23:27:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:27:56.561378 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-be4325ba-26b0-4918-9440-2962bc45a4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:56.562328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db084913-69f5-40a8-afd8-f884161f7fea None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.562955 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db084913-69f5-40a8-afd8-f884161f7fea None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.563510 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db084913-69f5-40a8-afd8-f884161f7fea None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.566105 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 817/1612] 10.222.0.27 () {54 vars in 962 bytes} [Mon Sep 12 23:27:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:27:56.567795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c716d-1f54-4c7d-b0ed-69919a344077 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.568276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c716d-1f54-4c7d-b0ed-69919a344077 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.568993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c716d-1f54-4c7d-b0ed-69919a344077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.594543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c39c716d-1f54-4c7d-b0ed-69919a344077 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:56.597110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c39c716d-1f54-4c7d-b0ed-69919a344077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:56.644104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:56.662317 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 796/1613] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:56 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:56.666179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4325ba-26b0-4918-9440-2962bc45a4a3 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:56.694571 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 818/1614] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:56.696569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e29baa-26ce-41b8-a20a-7bc4c48621f7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.697443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e29baa-26ce-41b8-a20a-7bc4c48621f7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.697917 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e29baa-26ce-41b8-a20a-7bc4c48621f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.723907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-69e29baa-26ce-41b8-a20a-7bc4c48621f7 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:56.725500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-69e29baa-26ce-41b8-a20a-7bc4c48621f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:56.735668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 819/1615] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:56.735899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fada7d8-3fca-4e3b-9297-f5716d2d9ea8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.736126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fada7d8-3fca-4e3b-9297-f5716d2d9ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.736306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fada7d8-3fca-4e3b-9297-f5716d2d9ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.772500 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:56.786808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4325ba-26b0-4918-9440-2962bc45a4a3 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:56.792165 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d7b9c-2ece-4055-a7d2-10fe81ec23b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:56.793155 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d7b9c-2ece-4055-a7d2-10fe81ec23b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.793155 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d7b9c-2ece-4055-a7d2-10fe81ec23b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.793361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0fada7d8-3fca-4e3b-9297-f5716d2d9ea8 None None] MFA Rules not processed for user `f5fa28ffdafd435cb2d93bd453add3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:56.793633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d7b9c-2ece-4055-a7d2-10fe81ec23b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.794848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0fada7d8-3fca-4e3b-9297-f5716d2d9ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:56.807669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d7b9c-2ece-4055-a7d2-10fe81ec23b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:56.809945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d7b9c-2ece-4055-a7d2-10fe81ec23b8 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:56.828662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 797/1616] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:27:56.830154 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421190e6-5968-48f0-bfa1-b02454fbe867 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.830450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421190e6-5968-48f0-bfa1-b02454fbe867 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.832094 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421190e6-5968-48f0-bfa1-b02454fbe867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.857823 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-421190e6-5968-48f0-bfa1-b02454fbe867 None None] MFA Rules not processed for user `f5fa28ffdafd435cb2d93bd453add3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:56.859112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-421190e6-5968-48f0-bfa1-b02454fbe867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:56.916894 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 820/1617] 10.222.0.27 () {58 vars in 1059 bytes} [Mon Sep 12 23:27:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:56.918121 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fada7d8-3fca-4e3b-9297-f5716d2d9ea8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:56.933138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fada7d8-3fca-4e3b-9297-f5716d2d9ea8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:56.940937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00023d-ece5-4dc0-aef1-01cca3c3ce26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:56.942300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00023d-ece5-4dc0-aef1-01cca3c3ce26 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:56.942300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00023d-ece5-4dc0-aef1-01cca3c3ce26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:56.942681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00023d-ece5-4dc0-aef1-01cca3c3ce26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:56.944046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00023d-ece5-4dc0-aef1-01cca3c3ce26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:56.946581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00023d-ece5-4dc0-aef1-01cca3c3ce26 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:56.982526 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 798/1618] 10.222.0.27 () {58 vars in 1059 bytes} [Mon Sep 12 23:27:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:56.983909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421190e6-5968-48f0-bfa1-b02454fbe867 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:56.985613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-421190e6-5968-48f0-bfa1-b02454fbe867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:56.989834 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 821/1619] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:27:56 2022] POST /identity/v3/projects => generated 405 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:56.991433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00023d-ece5-4dc0-aef1-01cca3c3ce26 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:56.993912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ef00023d-ece5-4dc0-aef1-01cca3c3ce26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:57.005704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421190e6-5968-48f0-bfa1-b02454fbe867 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:57.071582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00023d-ece5-4dc0-aef1-01cca3c3ce26 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:57.075320 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765428fb-adf9-4966-a8b1-3087ea1a0b37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:57.076029 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765428fb-adf9-4966-a8b1-3087ea1a0b37 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:57.076311 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765428fb-adf9-4966-a8b1-3087ea1a0b37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:57.076585 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765428fb-adf9-4966-a8b1-3087ea1a0b37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:57.079850 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab951e76-84e7-4659-8f08-a33afec0161e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:57.081467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab951e76-84e7-4659-8f08-a33afec0161e service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:57.081852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab951e76-84e7-4659-8f08-a33afec0161e service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:57.082351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab951e76-84e7-4659-8f08-a33afec0161e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:57.115259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:57.149028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765428fb-adf9-4966-a8b1-3087ea1a0b37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:57.150543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765428fb-adf9-4966-a8b1-3087ea1a0b37 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:57.166049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab951e76-84e7-4659-8f08-a33afec0161e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:57.168336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab951e76-84e7-4659-8f08-a33afec0161e service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:57.173366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 822/1620] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:27:57.178539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765428fb-adf9-4966-a8b1-3087ea1a0b37 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:57.200316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 799/1621] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:27:57.201509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab951e76-84e7-4659-8f08-a33afec0161e service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:57.244964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:57.303200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab951e76-84e7-4659-8f08-a33afec0161e service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:57.324241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175b2f8-cc49-4538-aa8d-5a0fd34cd12b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:57.325537 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175b2f8-cc49-4538-aa8d-5a0fd34cd12b service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:57.326884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175b2f8-cc49-4538-aa8d-5a0fd34cd12b service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:57.328227 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175b2f8-cc49-4538-aa8d-5a0fd34cd12b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:57.361137 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765428fb-adf9-4966-a8b1-3087ea1a0b37 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:57.367418 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd7e19-4d70-47b9-b7f6-2b910b915d33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:57.368512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd7e19-4d70-47b9-b7f6-2b910b915d33 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:57.368512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd7e19-4d70-47b9-b7f6-2b910b915d33 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:57.368714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd7e19-4d70-47b9-b7f6-2b910b915d33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:57.419561 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175b2f8-cc49-4538-aa8d-5a0fd34cd12b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:57.421878 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175b2f8-cc49-4538-aa8d-5a0fd34cd12b service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:57.433426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 800/1622] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:27:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:27:57.434375 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175b2f8-cc49-4538-aa8d-5a0fd34cd12b service nova] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:57.447056 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175b2f8-cc49-4538-aa8d-5a0fd34cd12b service nova] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:57.453475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25ae05-9b38-46e3-ae0f-da8b34eafca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:57.454012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25ae05-9b38-46e3-ae0f-da8b34eafca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:57.454298 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25ae05-9b38-46e3-ae0f-da8b34eafca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:57.454811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25ae05-9b38-46e3-ae0f-da8b34eafca7 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:57.455635 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25ae05-9b38-46e3-ae0f-da8b34eafca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:57.458267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25ae05-9b38-46e3-ae0f-da8b34eafca7 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:57.478109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd7e19-4d70-47b9-b7f6-2b910b915d33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:57.482099 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd7e19-4d70-47b9-b7f6-2b910b915d33 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:57.495266 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 801/1623] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:27:57 2022] POST /identity/v3/users => generated 309 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:27:57.496538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25ae05-9b38-46e3-ae0f-da8b34eafca7 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:57.504791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 823/1624] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:27:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:27:57.518483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25ae05-9b38-46e3-ae0f-da8b34eafca7 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:57.532416 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f492b1-bfe1-442c-9d57-3a82f92c86d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:57.535443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f492b1-bfe1-442c-9d57-3a82f92c86d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:57.535443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f492b1-bfe1-442c-9d57-3a82f92c86d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:57.535443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f492b1-bfe1-442c-9d57-3a82f92c86d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:57.535443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f492b1-bfe1-442c-9d57-3a82f92c86d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:57.617642 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f492b1-bfe1-442c-9d57-3a82f92c86d3 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:57.617642 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 802/1625] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:57.618360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd7e19-4d70-47b9-b7f6-2b910b915d33 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:57.618360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd7e19-4d70-47b9-b7f6-2b910b915d33 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:57.619160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ae61b-7ad7-420b-9d41-f022d83a499c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:57.620252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ae61b-7ad7-420b-9d41-f022d83a499c admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:57.620888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ae61b-7ad7-420b-9d41-f022d83a499c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:57.621360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ae61b-7ad7-420b-9d41-f022d83a499c admin admin] PATH_INFO: `/v3/projects/fa02fb9efd8a427d899e47866183d9bb/users/0dfe47453cc240b4ba7ab631c81945b0/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:57.668286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ae61b-7ad7-420b-9d41-f022d83a499c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa02fb9efd8a427d899e47866183d9bb, user_id=0dfe47453cc240b4ba7ab631c81945b0, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:57.676158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ae61b-7ad7-420b-9d41-f022d83a499c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:57.703416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 824/1626] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:27:57 2022] PUT /identity/v3/projects/fa02fb9efd8a427d899e47866183d9bb/users/0dfe47453cc240b4ba7ab631c81945b0/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:27:57.704717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f492b1-bfe1-442c-9d57-3a82f92c86d3 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:57.719598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f492b1-bfe1-442c-9d57-3a82f92c86d3 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:57.857288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93252e-a878-49ef-8489-1124c5450b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:57.857921 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93252e-a878-49ef-8489-1124c5450b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:57.858208 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93252e-a878-49ef-8489-1124c5450b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:57.858489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93252e-a878-49ef-8489-1124c5450b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:57.859140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93252e-a878-49ef-8489-1124c5450b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:57.860811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93252e-a878-49ef-8489-1124c5450b1b admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:57.884065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 803/1627] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:27:57.885788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ae61b-7ad7-420b-9d41-f022d83a499c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:57.899603 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ae61b-7ad7-420b-9d41-f022d83a499c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:57.899836 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce24c19-0b48-4c6b-a05c-2e33002116d2 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:57.900264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce24c19-0b48-4c6b-a05c-2e33002116d2 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:57.900685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce24c19-0b48-4c6b-a05c-2e33002116d2 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:57.902460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 804/1628] 10.222.0.27 () {54 vars in 912 bytes} [Mon Sep 12 23:27:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:27:57.909402 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3a6b5-5954-400c-9dff-44102dd42529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:57.909593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c843e1-8152-41f8-a8b9-95189ac7fcc8 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:57.909691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3a6b5-5954-400c-9dff-44102dd42529 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:57.909969 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3a6b5-5954-400c-9dff-44102dd42529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:57.910185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3a6b5-5954-400c-9dff-44102dd42529 admin admin] PATH_INFO: `/v3/projects/fa02fb9efd8a427d899e47866183d9bb/users/0dfe47453cc240b4ba7ab631c81945b0/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:57.917084 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3a6b5-5954-400c-9dff-44102dd42529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa02fb9efd8a427d899e47866183d9bb, user_id=0dfe47453cc240b4ba7ab631c81945b0, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:57.919491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c843e1-8152-41f8-a8b9-95189ac7fcc8 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:57.920086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c843e1-8152-41f8-a8b9-95189ac7fcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:57.923346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3a6b5-5954-400c-9dff-44102dd42529 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:57.944026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-e4c843e1-8152-41f8-a8b9-95189ac7fcc8 None None] MFA Rules not processed for user `f5fa28ffdafd435cb2d93bd453add3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:57.945426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e4c843e1-8152-41f8-a8b9-95189ac7fcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:57.948557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 825/1629] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:27:57 2022] PUT /identity/v3/projects/fa02fb9efd8a427d899e47866183d9bb/users/0dfe47453cc240b4ba7ab631c81945b0/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:27:57.960555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd46376-a5f4-4537-877e-ed1292a3dc5c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:57.961273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd46376-a5f4-4537-877e-ed1292a3dc5c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:57.961555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd46376-a5f4-4537-877e-ed1292a3dc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:58.013890 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:58.037809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5fd46376-a5f4-4537-877e-ed1292a3dc5c None None] MFA Rules not processed for user `0dfe47453cc240b4ba7ab631c81945b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:58.041005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5fd46376-a5f4-4537-877e-ed1292a3dc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:58.072206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:27:58.098118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 805/1630] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:27:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:27:58.099828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be65a64-cf60-4774-b9e5-7b18e8615279 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:58.100117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be65a64-cf60-4774-b9e5-7b18e8615279 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:58.100413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be65a64-cf60-4774-b9e5-7b18e8615279 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:58.102278 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 806/1631] 10.222.0.27 () {54 vars in 912 bytes} [Mon Sep 12 23:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:27:58.108562 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4db70-9471-4e0e-be17-15486c166de5 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:58.108842 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4db70-9471-4e0e-be17-15486c166de5 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:58.109170 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4db70-9471-4e0e-be17-15486c166de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:58.139874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-bcd4db70-9471-4e0e-be17-15486c166de5 None None] MFA Rules not processed for user `f5fa28ffdafd435cb2d93bd453add3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:27:58.141325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bcd4db70-9471-4e0e-be17-15486c166de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:27:58.160012 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 826/1632] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:27:57 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:27:58.161013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd46376-a5f4-4537-877e-ed1292a3dc5c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:58.162476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 807/1633] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:27:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:27:58.178710 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4db70-9471-4e0e-be17-15486c166de5 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:58.265953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd46376-a5f4-4537-877e-ed1292a3dc5c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:58.271316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaee491-0005-4ae0-9f36-86dd70641c6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:58.272749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaee491-0005-4ae0-9f36-86dd70641c6b service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:58.272749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaee491-0005-4ae0-9f36-86dd70641c6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:58.272749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaee491-0005-4ae0-9f36-86dd70641c6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:58.273334 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4db70-9471-4e0e-be17-15486c166de5 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:58.279927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec311f5-4184-4bea-bdb6-e83238a1dd91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:58.286157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaee491-0005-4ae0-9f36-86dd70641c6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:58.286414 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec311f5-4184-4bea-bdb6-e83238a1dd91 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:58.287987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaee491-0005-4ae0-9f36-86dd70641c6b service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:58.288131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec311f5-4184-4bea-bdb6-e83238a1dd91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:58.288225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec311f5-4184-4bea-bdb6-e83238a1dd91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:58.307969 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 827/1634] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:27:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:27:58.313138 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec311f5-4184-4bea-bdb6-e83238a1dd91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:58.315959 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec311f5-4184-4bea-bdb6-e83238a1dd91 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:58.353101 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 808/1635] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:27:58 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:27:58.465285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaee491-0005-4ae0-9f36-86dd70641c6b service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:58.474939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaee491-0005-4ae0-9f36-86dd70641c6b service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:58.489196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a91de-c870-4718-998c-a7eb0f435a7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:58.490175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a91de-c870-4718-998c-a7eb0f435a7c service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:58.490538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a91de-c870-4718-998c-a7eb0f435a7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:58.490885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a91de-c870-4718-998c-a7eb0f435a7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:58.519333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a91de-c870-4718-998c-a7eb0f435a7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:58.523301 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a91de-c870-4718-998c-a7eb0f435a7c service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:27:58.556554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 828/1636] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:27:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:27:59.872813 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec311f5-4184-4bea-bdb6-e83238a1dd91 service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:27:59.964851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec311f5-4184-4bea-bdb6-e83238a1dd91 service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:27:59.969742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:27:59.970390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:27:59.970673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:27:59.970954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] PATH_INFO: `/v3/users/17715e1c91a246489280bf04ed7fae0b` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:27:59.972159 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17715e1c91a246489280bf04ed7fae0b)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:27:59.973194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: WARNING py.warnings [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:27:59.973194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: warnings.warn(msg) Sep 12 23:27:59.973194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: Sep 12 23:27:59.974405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:00.055690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17715e1c91a246489280bf04ed7fae0b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:00.082738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17715e1c91a246489280bf04ed7fae0b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:00.084485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:00.092239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17715e1c91a246489280bf04ed7fae0b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:00.093141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17715e1c91a246489280bf04ed7fae0b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:00.105967 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17715e1c91a246489280bf04ed7fae0b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:00.124892 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 809/1637] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:27:59 2022] DELETE /identity/v3/users/17715e1c91a246489280bf04ed7fae0b => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:00.129621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cf9c1-b8f0-406e-a237-20742f6e0bfb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:00.130580 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cf9c1-b8f0-406e-a237-20742f6e0bfb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:00.131057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cf9c1-b8f0-406e-a237-20742f6e0bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:00.163668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-079cf9c1-b8f0-406e-a237-20742f6e0bfb None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:00.165234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-079cf9c1-b8f0-406e-a237-20742f6e0bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:00.262309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 829/1638] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:00.284307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:00.286481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:00.402614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3269e7-0a77-4c16-9030-768e4402c352 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:00.407760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e3901-b9df-4961-bd8e-3f2dfc6da09a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:00.409325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e3901-b9df-4961-bd8e-3f2dfc6da09a service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:00.409716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e3901-b9df-4961-bd8e-3f2dfc6da09a service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:00.410094 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e3901-b9df-4961-bd8e-3f2dfc6da09a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:00.425819 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e3901-b9df-4961-bd8e-3f2dfc6da09a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:00.427820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e3901-b9df-4961-bd8e-3f2dfc6da09a service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:00.445606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 810/1639] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:00.531188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba2637-aaf7-4816-9aad-be3ebf016ba9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:00.532235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba2637-aaf7-4816-9aad-be3ebf016ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:00.532673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba2637-aaf7-4816-9aad-be3ebf016ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:00.621017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-94ba2637-aaf7-4816-9aad-be3ebf016ba9 None None] MFA Rules not processed for user `a29f8dda51c44e08a46fabe3fe9f86ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:00.622690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-94ba2637-aaf7-4816-9aad-be3ebf016ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:00.712821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:00.799025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 830/1640] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:00 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:00.801036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e3901-b9df-4961-bd8e-3f2dfc6da09a service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:00.802537 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c77e3901-b9df-4961-bd8e-3f2dfc6da09a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:00.828570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e3901-b9df-4961-bd8e-3f2dfc6da09a service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:00.841702 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02862c-6e4b-40dd-9d2c-19dcd21ac3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:00.842173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02862c-6e4b-40dd-9d2c-19dcd21ac3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:00.842459 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02862c-6e4b-40dd-9d2c-19dcd21ac3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:00.842732 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02862c-6e4b-40dd-9d2c-19dcd21ac3e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:00.843385 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02862c-6e4b-40dd-9d2c-19dcd21ac3e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:00.847898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02862c-6e4b-40dd-9d2c-19dcd21ac3e6 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:00.921111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 811/1641] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:00 2022] POST /identity/v3/projects => generated 403 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:00.922607 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba2637-aaf7-4816-9aad-be3ebf016ba9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:00.937840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba2637-aaf7-4816-9aad-be3ebf016ba9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:01.084657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f11082-914b-4347-8ce1-4f74d5bebd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:01.084912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f11082-914b-4347-8ce1-4f74d5bebd44 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:01.085052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f11082-914b-4347-8ce1-4f74d5bebd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:01.085285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f11082-914b-4347-8ce1-4f74d5bebd44 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:01.090841 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f11082-914b-4347-8ce1-4f74d5bebd44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:01.094177 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f11082-914b-4347-8ce1-4f74d5bebd44 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:01.153279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 831/1642] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:00 2022] POST /identity/v3/users => generated 308 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:01.157558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02862c-6e4b-40dd-9d2c-19dcd21ac3e6 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:01.176284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02862c-6e4b-40dd-9d2c-19dcd21ac3e6 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:01.190283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e66ffc-625c-4258-b7b3-f11a9a383af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:01.190914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e66ffc-625c-4258-b7b3-f11a9a383af7 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:01.191245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e66ffc-625c-4258-b7b3-f11a9a383af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:01.191637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e66ffc-625c-4258-b7b3-f11a9a383af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:01.192362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e66ffc-625c-4258-b7b3-f11a9a383af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:01.195684 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e66ffc-625c-4258-b7b3-f11a9a383af7 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:01.228438 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 812/1643] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:01.231632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f11082-914b-4347-8ce1-4f74d5bebd44 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:01.249211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f11082-914b-4347-8ce1-4f74d5bebd44 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:01.262807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d1b55-d3b0-4125-9757-24007b10dc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:01.263495 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d1b55-d3b0-4125-9757-24007b10dc7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:01.263874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d1b55-d3b0-4125-9757-24007b10dc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:01.264152 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d1b55-d3b0-4125-9757-24007b10dc7a admin admin] PATH_INFO: `/v3/projects/370e5966bb6a4eda8567bad550f443f2/users/04017f4b4b0547719538ae7befa0ea17/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:01.301769 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e66ffc-625c-4258-b7b3-f11a9a383af7 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:01.305450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d1b55-d3b0-4125-9757-24007b10dc7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=370e5966bb6a4eda8567bad550f443f2, user_id=04017f4b4b0547719538ae7befa0ea17, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:01.308595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e7e66ffc-625c-4258-b7b3-f11a9a383af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:01.335406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d1b55-d3b0-4125-9757-24007b10dc7a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:01.346871 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e66ffc-625c-4258-b7b3-f11a9a383af7 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:01.357625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:01.358284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:01.362226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:01.366180 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] PATH_INFO: `/v3/projects/45befad61bc94b1aafb57766cfad5be4` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:01.376963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=45befad61bc94b1aafb57766cfad5be4)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:01.379055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 832/1644] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:01 2022] PUT /identity/v3/projects/370e5966bb6a4eda8567bad550f443f2/users/04017f4b4b0547719538ae7befa0ea17/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:01.379321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:01.383120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d1b55-d3b0-4125-9757-24007b10dc7a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:01.540023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45befad61bc94b1aafb57766cfad5be4', 'request_id': 'req-5376968f-23f0-49ec-bdc8-48eef8821ddb'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:01.554035 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45befad61bc94b1aafb57766cfad5be4', 'request_id': 'req-5376968f-23f0-49ec-bdc8-48eef8821ddb'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:01.570715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45befad61bc94b1aafb57766cfad5be4', 'request_id': 'req-5376968f-23f0-49ec-bdc8-48eef8821ddb'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:01.579558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d1b55-d3b0-4125-9757-24007b10dc7a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:01.595867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] The token cache is being invalidate because project 45befad61bc94b1aafb57766cfad5be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:01.597787 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:01.599856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:01.607892 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 813/1645] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:01 2022] DELETE /identity/v3/projects/45befad61bc94b1aafb57766cfad5be4 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:28:01.612936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:01.614645 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:01.675848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:01.707071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:01.731555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27053212-085a-45dd-a30c-9e699cd227f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:01.733251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27053212-085a-45dd-a30c-9e699cd227f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:01.733540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27053212-085a-45dd-a30c-9e699cd227f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:01.735150 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27053212-085a-45dd-a30c-9e699cd227f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:01.737587 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27053212-085a-45dd-a30c-9e699cd227f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:01.741332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27053212-085a-45dd-a30c-9e699cd227f1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:01.758264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376968f-23f0-49ec-bdc8-48eef8821ddb admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:01.764072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c9f1e-2b06-46ff-b5b6-f4d9b383955b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:01.764648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c9f1e-2b06-46ff-b5b6-f4d9b383955b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:01.764852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c9f1e-2b06-46ff-b5b6-f4d9b383955b admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:01.765131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c9f1e-2b06-46ff-b5b6-f4d9b383955b admin admin] PATH_INFO: `/v3/users/0dfe47453cc240b4ba7ab631c81945b0` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:01.766264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c9f1e-2b06-46ff-b5b6-f4d9b383955b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dfe47453cc240b4ba7ab631c81945b0)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:01.769578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c9f1e-2b06-46ff-b5b6-f4d9b383955b admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:01.774677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 833/1646] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:28:01.783759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27053212-085a-45dd-a30c-9e699cd227f1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:01.786149 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-27053212-085a-45dd-a30c-9e699cd227f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:01.803663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27053212-085a-45dd-a30c-9e699cd227f1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:01.810495 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fdbf9-5248-4e51-97bf-1d48c565a318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:01.811177 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fdbf9-5248-4e51-97bf-1d48c565a318 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:01.811455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fdbf9-5248-4e51-97bf-1d48c565a318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:01.811771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fdbf9-5248-4e51-97bf-1d48c565a318 admin admin] PATH_INFO: `/v3/projects/370e5966bb6a4eda8567bad550f443f2/users/04017f4b4b0547719538ae7befa0ea17/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:01.814459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fdbf9-5248-4e51-97bf-1d48c565a318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=370e5966bb6a4eda8567bad550f443f2, user_id=04017f4b4b0547719538ae7befa0ea17, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:01.819646 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fdbf9-5248-4e51-97bf-1d48c565a318 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:01.836166 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 834/1647] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:01 2022] PUT /identity/v3/projects/370e5966bb6a4eda8567bad550f443f2/users/04017f4b4b0547719538ae7befa0ea17/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:01.882253 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9b3c9f1e-2b06-46ff-b5b6-f4d9b383955b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dfe47453cc240b4ba7ab631c81945b0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:01.902440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9b3c9f1e-2b06-46ff-b5b6-f4d9b383955b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dfe47453cc240b4ba7ab631c81945b0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:01.914402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9b3c9f1e-2b06-46ff-b5b6-f4d9b383955b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dfe47453cc240b4ba7ab631c81945b0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:01.918215 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9b3c9f1e-2b06-46ff-b5b6-f4d9b383955b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dfe47453cc240b4ba7ab631c81945b0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:01.938840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9b3c9f1e-2b06-46ff-b5b6-f4d9b383955b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dfe47453cc240b4ba7ab631c81945b0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:01.952616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 814/1648] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:01 2022] DELETE /identity/v3/users/0dfe47453cc240b4ba7ab631c81945b0 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:02.420616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fdbf9-5248-4e51-97bf-1d48c565a318 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:02.422383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f48fdbf9-5248-4e51-97bf-1d48c565a318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:02.533740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fdbf9-5248-4e51-97bf-1d48c565a318 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:02.621308 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1f545-cb9b-4661-9930-20801e39eea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:02.622116 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1f545-cb9b-4661-9930-20801e39eea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:02.622919 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1f545-cb9b-4661-9930-20801e39eea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:02.623067 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1f545-cb9b-4661-9930-20801e39eea6 admin admin] PATH_INFO: `/v3/users/582ac771652d49f7a79facf89374e5bf` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:02.625709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1f545-cb9b-4661-9930-20801e39eea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=582ac771652d49f7a79facf89374e5bf)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:02.628661 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1f545-cb9b-4661-9930-20801e39eea6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:02.722749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6dd1f545-cb9b-4661-9930-20801e39eea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '582ac771652d49f7a79facf89374e5bf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:02.743612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6dd1f545-cb9b-4661-9930-20801e39eea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '582ac771652d49f7a79facf89374e5bf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:02.767411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6dd1f545-cb9b-4661-9930-20801e39eea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '582ac771652d49f7a79facf89374e5bf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:02.782095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6dd1f545-cb9b-4661-9930-20801e39eea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '582ac771652d49f7a79facf89374e5bf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:02.783211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6dd1f545-cb9b-4661-9930-20801e39eea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '582ac771652d49f7a79facf89374e5bf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:02.786730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:02.795333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 835/1649] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:02 2022] DELETE /identity/v3/users/582ac771652d49f7a79facf89374e5bf => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:02.802039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a1b46-6253-4f72-b033-fa12c12537d7 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:02.802631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a1b46-6253-4f72-b033-fa12c12537d7 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:02.802923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a1b46-6253-4f72-b033-fa12c12537d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:02.831438 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-739a1b46-6253-4f72-b033-fa12c12537d7 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:02.833696 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-739a1b46-6253-4f72-b033-fa12c12537d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:02.979327 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 815/1650] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:03.081561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1f545-cb9b-4661-9930-20801e39eea6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:03.083294 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6dd1f545-cb9b-4661-9930-20801e39eea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:03.224233 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1f545-cb9b-4661-9930-20801e39eea6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:03.262636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d61a82-9c0b-42ff-ac0b-38dc581fe180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:03.264028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d61a82-9c0b-42ff-ac0b-38dc581fe180 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:03.265637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d61a82-9c0b-42ff-ac0b-38dc581fe180 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:03.268764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d61a82-9c0b-42ff-ac0b-38dc581fe180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:03.297553 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d61a82-9c0b-42ff-ac0b-38dc581fe180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:03.299719 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d61a82-9c0b-42ff-ac0b-38dc581fe180 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:03.314407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 836/1651] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:28:03.685513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a1b46-6253-4f72-b033-fa12c12537d7 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:03.687510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-739a1b46-6253-4f72-b033-fa12c12537d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:03.708526 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a1b46-6253-4f72-b033-fa12c12537d7 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:03.730852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de6e28-bc25-4908-ac05-5bd0292d05cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:03.731940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de6e28-bc25-4908-ac05-5bd0292d05cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:03.739628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de6e28-bc25-4908-ac05-5bd0292d05cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:03.740140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de6e28-bc25-4908-ac05-5bd0292d05cf admin admin] PATH_INFO: `/v3/projects/fa02fb9efd8a427d899e47866183d9bb` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:03.743742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de6e28-bc25-4908-ac05-5bd0292d05cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa02fb9efd8a427d899e47866183d9bb)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:03.752660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de6e28-bc25-4908-ac05-5bd0292d05cf admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:03.897093 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-16de6e28-bc25-4908-ac05-5bd0292d05cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa02fb9efd8a427d899e47866183d9bb', 'request_id': 'req-16de6e28-bc25-4908-ac05-5bd0292d05cf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:03.899715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:03.914016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-16de6e28-bc25-4908-ac05-5bd0292d05cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa02fb9efd8a427d899e47866183d9bb', 'request_id': 'req-16de6e28-bc25-4908-ac05-5bd0292d05cf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:03.934876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-16de6e28-bc25-4908-ac05-5bd0292d05cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa02fb9efd8a427d899e47866183d9bb', 'request_id': 'req-16de6e28-bc25-4908-ac05-5bd0292d05cf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:03.971269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-16de6e28-bc25-4908-ac05-5bd0292d05cf admin admin] The token cache is being invalidate because project fa02fb9efd8a427d899e47866183d9bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:03.971931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-16de6e28-bc25-4908-ac05-5bd0292d05cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:03.974341 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-16de6e28-bc25-4908-ac05-5bd0292d05cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:03.980428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 816/1652] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:03 2022] DELETE /identity/v3/projects/fa02fb9efd8a427d899e47866183d9bb => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:03.981906 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051dd1-a675-40e8-9e79-03beff3f77f6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:03.982308 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051dd1-a675-40e8-9e79-03beff3f77f6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:03.982517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051dd1-a675-40e8-9e79-03beff3f77f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:04.049560 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-14051dd1-a675-40e8-9e79-03beff3f77f6 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:04.062504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-14051dd1-a675-40e8-9e79-03beff3f77f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:04.244500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 837/1653] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:28:04.245219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bc977-6bc9-4f3e-8ab0-2decbd8f34b7 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:04.245507 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bc977-6bc9-4f3e-8ab0-2decbd8f34b7 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:04.245796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bc977-6bc9-4f3e-8ab0-2decbd8f34b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:04.336135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-368bc977-6bc9-4f3e-8ab0-2decbd8f34b7 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:04.337440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-368bc977-6bc9-4f3e-8ab0-2decbd8f34b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:04.382597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 817/1654] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:04.386827 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14051dd1-a675-40e8-9e79-03beff3f77f6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:04.414963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14051dd1-a675-40e8-9e79-03beff3f77f6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:04.441513 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3408466-8187-47c0-aa9f-025941ea64e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:04.442504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3408466-8187-47c0-aa9f-025941ea64e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:04.443419 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3408466-8187-47c0-aa9f-025941ea64e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:04.443963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3408466-8187-47c0-aa9f-025941ea64e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:04.445087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3408466-8187-47c0-aa9f-025941ea64e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:28:04.457248 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3408466-8187-47c0-aa9f-025941ea64e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:04.463508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3408466-8187-47c0-aa9f-025941ea64e0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:04.565951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 838/1655] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:28:04.567587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368bc977-6bc9-4f3e-8ab0-2decbd8f34b7 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:04.592884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368bc977-6bc9-4f3e-8ab0-2decbd8f34b7 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:04.616046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f3ccd-cd70-4710-950c-f86be0fe6088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:04.617971 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3ccd-cd70-4710-950c-f86be0fe6088 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:04.618399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3ccd-cd70-4710-950c-f86be0fe6088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:04.619116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3ccd-cd70-4710-950c-f86be0fe6088 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:04.619988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f3ccd-cd70-4710-950c-f86be0fe6088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:04.626509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f3ccd-cd70-4710-950c-f86be0fe6088 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:04.702873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 818/1656] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:04 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:04.708332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3408466-8187-47c0-aa9f-025941ea64e0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:04.727261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3408466-8187-47c0-aa9f-025941ea64e0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:04.854593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfce0dc-d2a8-4bbe-a8a3-237c785355c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:04.856681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfce0dc-d2a8-4bbe-a8a3-237c785355c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:04.857185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfce0dc-d2a8-4bbe-a8a3-237c785355c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:04.857823 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfce0dc-d2a8-4bbe-a8a3-237c785355c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:04.858337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfce0dc-d2a8-4bbe-a8a3-237c785355c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:04.865363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfce0dc-d2a8-4bbe-a8a3-237c785355c8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:04.912304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 839/1657] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:04 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:04.916413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f3ccd-cd70-4710-950c-f86be0fe6088 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:04.929582 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f3ccd-cd70-4710-950c-f86be0fe6088 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:04.939537 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2516a3b-be6b-44fa-9a0d-e450beba0265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:04.940984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2516a3b-be6b-44fa-9a0d-e450beba0265 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:04.941472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2516a3b-be6b-44fa-9a0d-e450beba0265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:04.941937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2516a3b-be6b-44fa-9a0d-e450beba0265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:04.943039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2516a3b-be6b-44fa-9a0d-e450beba0265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:04.946514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2516a3b-be6b-44fa-9a0d-e450beba0265 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:04.965856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 819/1658] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:04.970465 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfce0dc-d2a8-4bbe-a8a3-237c785355c8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:04.986657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfce0dc-d2a8-4bbe-a8a3-237c785355c8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:05.001809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ac2f1-e03d-45e2-835d-bace2885dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:05.003100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ac2f1-e03d-45e2-835d-bace2885dcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:05.003480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ac2f1-e03d-45e2-835d-bace2885dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:05.003947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ac2f1-e03d-45e2-835d-bace2885dcf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:05.005087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ac2f1-e03d-45e2-835d-bace2885dcf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:05.007875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ac2f1-e03d-45e2-835d-bace2885dcf2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:05.026342 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 840/1659] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:05.030369 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2516a3b-be6b-44fa-9a0d-e450beba0265 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:05.046611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2516a3b-be6b-44fa-9a0d-e450beba0265 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:05.059306 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bc511-1be6-4d1a-bcf3-e301a83f3b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:05.060062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bc511-1be6-4d1a-bcf3-e301a83f3b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:05.060344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bc511-1be6-4d1a-bcf3-e301a83f3b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:05.060621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758bc511-1be6-4d1a-bcf3-e301a83f3b37 admin admin] PATH_INFO: `/v3/projects/1f606e6b59bd48daad82fdf38e01ae4d/users/34c651a6a1a140eebf4914f521756552/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:05.103695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758bc511-1be6-4d1a-bcf3-e301a83f3b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f606e6b59bd48daad82fdf38e01ae4d, user_id=34c651a6a1a140eebf4914f521756552, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:05.110722 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758bc511-1be6-4d1a-bcf3-e301a83f3b37 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:05.145312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 820/1660] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:05 2022] PUT /identity/v3/projects/1f606e6b59bd48daad82fdf38e01ae4d/users/34c651a6a1a140eebf4914f521756552/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:05.155533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510de8b-3913-476e-bb01-b8cff94454f5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:05.156122 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510de8b-3913-476e-bb01-b8cff94454f5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:05.156382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510de8b-3913-476e-bb01-b8cff94454f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:05.176699 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:05.216955 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a510de8b-3913-476e-bb01-b8cff94454f5 None None] MFA Rules not processed for user `34c651a6a1a140eebf4914f521756552`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:05.218230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a510de8b-3913-476e-bb01-b8cff94454f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:05.339059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 841/1661] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:05.343281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bc511-1be6-4d1a-bcf3-e301a83f3b37 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:05.531013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758bc511-1be6-4d1a-bcf3-e301a83f3b37 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:05.543503 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03de584-9b52-420f-9228-2801e73d4312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:05.545610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03de584-9b52-420f-9228-2801e73d4312 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:05.546063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03de584-9b52-420f-9228-2801e73d4312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:05.548656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03de584-9b52-420f-9228-2801e73d4312 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:05.550688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03de584-9b52-420f-9228-2801e73d4312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:05.558105 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03de584-9b52-420f-9228-2801e73d4312 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:05.615314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 821/1662] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:05 2022] POST /identity/v3/projects => generated 377 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:05.619542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510de8b-3913-476e-bb01-b8cff94454f5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:05.637430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510de8b-3913-476e-bb01-b8cff94454f5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:05.791151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609e33a-e324-4e07-bac7-aefd6a86a039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:05.795220 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609e33a-e324-4e07-bac7-aefd6a86a039 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:05.795541 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609e33a-e324-4e07-bac7-aefd6a86a039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:05.795848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609e33a-e324-4e07-bac7-aefd6a86a039 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:05.796542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609e33a-e324-4e07-bac7-aefd6a86a039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:05.805309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609e33a-e324-4e07-bac7-aefd6a86a039 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:05.857081 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 842/1663] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:05 2022] POST /identity/v3/users => generated 295 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:28:05.859420 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03de584-9b52-420f-9228-2801e73d4312 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:05.876192 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03de584-9b52-420f-9228-2801e73d4312 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:05.882231 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134d7b0-7870-4a47-b300-d8847f2f14ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:05.883830 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134d7b0-7870-4a47-b300-d8847f2f14ef admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:05.884316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134d7b0-7870-4a47-b300-d8847f2f14ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:05.884795 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134d7b0-7870-4a47-b300-d8847f2f14ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:05.885671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8134d7b0-7870-4a47-b300-d8847f2f14ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:05.890650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8134d7b0-7870-4a47-b300-d8847f2f14ef admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:05.925969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 822/1664] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:05.930039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609e33a-e324-4e07-bac7-aefd6a86a039 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:05.946957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609e33a-e324-4e07-bac7-aefd6a86a039 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:05.952156 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d742634-b008-48d6-a3a9-af286ba5f157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:05.952857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d742634-b008-48d6-a3a9-af286ba5f157 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:05.953110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d742634-b008-48d6-a3a9-af286ba5f157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:05.953440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d742634-b008-48d6-a3a9-af286ba5f157 admin admin] PATH_INFO: `/v3/projects/23fc656bfad747739691f0b18e91e331/users/0fcabc07bca14f80a7329d876a35e30b/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:05.978686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d742634-b008-48d6-a3a9-af286ba5f157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23fc656bfad747739691f0b18e91e331, user_id=0fcabc07bca14f80a7329d876a35e30b, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:05.987826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d742634-b008-48d6-a3a9-af286ba5f157 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:06.010431 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 843/1665] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:05 2022] PUT /identity/v3/projects/23fc656bfad747739691f0b18e91e331/users/0fcabc07bca14f80a7329d876a35e30b/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:06.014440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134d7b0-7870-4a47-b300-d8847f2f14ef admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:06.035723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134d7b0-7870-4a47-b300-d8847f2f14ef admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:06.162363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:06.186260 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb879a-7ac7-41b8-87ad-a91d8dd3490e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:06.186902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb879a-7ac7-41b8-87ad-a91d8dd3490e admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:06.187185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb879a-7ac7-41b8-87ad-a91d8dd3490e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:06.187460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb879a-7ac7-41b8-87ad-a91d8dd3490e admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:06.190710 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb879a-7ac7-41b8-87ad-a91d8dd3490e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:06.194929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb879a-7ac7-41b8-87ad-a91d8dd3490e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:06.220520 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d742634-b008-48d6-a3a9-af286ba5f157 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:06.221583 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 823/1666] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:06.233251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d742634-b008-48d6-a3a9-af286ba5f157 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:06.250772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea14564-d2ef-4f18-ae08-02a377ac4bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:06.251461 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea14564-d2ef-4f18-ae08-02a377ac4bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:06.251769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea14564-d2ef-4f18-ae08-02a377ac4bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:06.252118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea14564-d2ef-4f18-ae08-02a377ac4bf2 admin admin] PATH_INFO: `/v3/projects/23fc656bfad747739691f0b18e91e331/users/0fcabc07bca14f80a7329d876a35e30b/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:06.257804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea14564-d2ef-4f18-ae08-02a377ac4bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23fc656bfad747739691f0b18e91e331, user_id=0fcabc07bca14f80a7329d876a35e30b, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:06.261126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea14564-d2ef-4f18-ae08-02a377ac4bf2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:06.275631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 844/1667] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:06 2022] PUT /identity/v3/projects/23fc656bfad747739691f0b18e91e331/users/0fcabc07bca14f80a7329d876a35e30b/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:28:06.288582 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bf3e7-81a2-4a0d-a6c4-601ebfedba46 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:06.288899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bf3e7-81a2-4a0d-a6c4-601ebfedba46 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:06.289133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bf3e7-81a2-4a0d-a6c4-601ebfedba46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:06.403150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-5b6bf3e7-81a2-4a0d-a6c4-601ebfedba46 None None] MFA Rules not processed for user `0fcabc07bca14f80a7329d876a35e30b`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:06.404932 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5b6bf3e7-81a2-4a0d-a6c4-601ebfedba46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:06.594866 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 824/1668] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:06.605538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea14564-d2ef-4f18-ae08-02a377ac4bf2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:06.608508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3ea14564-d2ef-4f18-ae08-02a377ac4bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:06.825337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:06.831067 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea14564-d2ef-4f18-ae08-02a377ac4bf2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:06.839630 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1117917-ec79-4bf9-a39f-ff07b755f295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:06.840649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1117917-ec79-4bf9-a39f-ff07b755f295 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:06.841318 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1117917-ec79-4bf9-a39f-ff07b755f295 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:06.841765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1117917-ec79-4bf9-a39f-ff07b755f295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:07.006063 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1117917-ec79-4bf9-a39f-ff07b755f295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:07.013577 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1117917-ec79-4bf9-a39f-ff07b755f295 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:07.042234 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f97a3c-674c-4f1c-a1b4-ad88ca9eb904 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:07.043526 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f97a3c-674c-4f1c-a1b4-ad88ca9eb904 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:07.044260 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f97a3c-674c-4f1c-a1b4-ad88ca9eb904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:07.069180 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 845/1669] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:07.103767 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-30f97a3c-674c-4f1c-a1b4-ad88ca9eb904 None None] MFA Rules not processed for user `04017f4b4b0547719538ae7befa0ea17`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:07.105549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-30f97a3c-674c-4f1c-a1b4-ad88ca9eb904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:07.215220 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 825/1670] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:07 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:28:07.331907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea450a-8bc8-4dcf-bb03-2b78133ff9b3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:07.332240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea450a-8bc8-4dcf-bb03-2b78133ff9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:07.332664 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea450a-8bc8-4dcf-bb03-2b78133ff9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:07.374119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4bea450a-8bc8-4dcf-bb03-2b78133ff9b3 None None] MFA Rules not processed for user `2749a61b3dd04d89a5c2e4235aa96934`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:07.375921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4bea450a-8bc8-4dcf-bb03-2b78133ff9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:07.517210 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 846/1671] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:07.538553 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f97a3c-674c-4f1c-a1b4-ad88ca9eb904 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:07.541424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-30f97a3c-674c-4f1c-a1b4-ad88ca9eb904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:07.641586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea450a-8bc8-4dcf-bb03-2b78133ff9b3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:07.645865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4bea450a-8bc8-4dcf-bb03-2b78133ff9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:07.678376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:07.704768 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f97a3c-674c-4f1c-a1b4-ad88ca9eb904 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:07.714394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c941f-8c32-497e-abed-7f62261e8e0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:07.715936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c941f-8c32-497e-abed-7f62261e8e0e service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:07.716588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c941f-8c32-497e-abed-7f62261e8e0e service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:07.717115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c941f-8c32-497e-abed-7f62261e8e0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:07.743804 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c941f-8c32-497e-abed-7f62261e8e0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:07.745458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c941f-8c32-497e-abed-7f62261e8e0e service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:07.756046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 826/1672] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:07.761830 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea450a-8bc8-4dcf-bb03-2b78133ff9b3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:07.766908 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:07.768088 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:07.768992 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:07.768992 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475 admin admin] PATH_INFO: `/v3/projects/1b1b14efb0dd42fcad321d9b5c8dafed` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:07.769987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b1b14efb0dd42fcad321d9b5c8dafed)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:07.771010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:28:07.771010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 12 23:28:07.771010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: Sep 12 23:28:07.772221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:07.832482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:07.862529 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b1b14efb0dd42fcad321d9b5c8dafed', 'request_id': 'req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:07.873076 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b1b14efb0dd42fcad321d9b5c8dafed', 'request_id': 'req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:07.888134 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b1b14efb0dd42fcad321d9b5c8dafed', 'request_id': 'req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:07.922552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475 admin admin] The token cache is being invalidate because project 1b1b14efb0dd42fcad321d9b5c8dafed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:07.922889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:07.923615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d9960a8-dc3e-49ff-9ab3-3a0173eeb475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:07.926804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 847/1673] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:07 2022] DELETE /identity/v3/projects/1b1b14efb0dd42fcad321d9b5c8dafed => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:07.931067 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abaf86-113b-4554-8373-5dbd63c7ede2 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:07.931431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abaf86-113b-4554-8373-5dbd63c7ede2 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:07.931824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abaf86-113b-4554-8373-5dbd63c7ede2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:07.970736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-12abaf86-113b-4554-8373-5dbd63c7ede2 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:07.972776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-12abaf86-113b-4554-8373-5dbd63c7ede2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:08.139914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 827/1674] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:28:08.148295 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399698e8-7b8d-4c1b-b4b3-402108618829 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:08.148837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399698e8-7b8d-4c1b-b4b3-402108618829 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:08.148948 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399698e8-7b8d-4c1b-b4b3-402108618829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:08.188408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-399698e8-7b8d-4c1b-b4b3-402108618829 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:08.190569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-399698e8-7b8d-4c1b-b4b3-402108618829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:08.218411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 848/1675] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:28:08.223302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abaf86-113b-4554-8373-5dbd63c7ede2 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:08.241901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abaf86-113b-4554-8373-5dbd63c7ede2 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:08.254621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0964025-8d85-44ad-9684-90fe32726af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:08.255257 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0964025-8d85-44ad-9684-90fe32726af1 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:08.255519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0964025-8d85-44ad-9684-90fe32726af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:08.255862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0964025-8d85-44ad-9684-90fe32726af1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:08.256466 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0964025-8d85-44ad-9684-90fe32726af1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:28:08.256770 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0964025-8d85-44ad-9684-90fe32726af1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:08.262002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0964025-8d85-44ad-9684-90fe32726af1 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:08.282884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399698e8-7b8d-4c1b-b4b3-402108618829 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:08.285123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-399698e8-7b8d-4c1b-b4b3-402108618829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:08.306451 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399698e8-7b8d-4c1b-b4b3-402108618829 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:08.327351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 828/1676] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:28:08.331947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9de35-2e3d-43da-a110-983c8cd16d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:08.333291 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0964025-8d85-44ad-9684-90fe32726af1 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:08.334093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9de35-2e3d-43da-a110-983c8cd16d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:08.334917 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9de35-2e3d-43da-a110-983c8cd16d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:08.342158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9de35-2e3d-43da-a110-983c8cd16d28 admin admin] PATH_INFO: `/v3/users/32e5b36aa97747d798b79509f3f66dd5` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:08.344178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9de35-2e3d-43da-a110-983c8cd16d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32e5b36aa97747d798b79509f3f66dd5)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:08.346670 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9de35-2e3d-43da-a110-983c8cd16d28 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:08.359014 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0964025-8d85-44ad-9684-90fe32726af1 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:08.373343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da278f-4284-4c91-8c98-3b7b15365136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:08.373968 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da278f-4284-4c91-8c98-3b7b15365136 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:08.374330 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da278f-4284-4c91-8c98-3b7b15365136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:08.374507 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da278f-4284-4c91-8c98-3b7b15365136 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:08.382538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da278f-4284-4c91-8c98-3b7b15365136 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:08.384717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da278f-4284-4c91-8c98-3b7b15365136 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:08.459211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7ee9de35-2e3d-43da-a110-983c8cd16d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32e5b36aa97747d798b79509f3f66dd5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:08.472137 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 829/1677] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:08 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:08.474902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da278f-4284-4c91-8c98-3b7b15365136 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:08.478137 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7ee9de35-2e3d-43da-a110-983c8cd16d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32e5b36aa97747d798b79509f3f66dd5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:08.520731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7ee9de35-2e3d-43da-a110-983c8cd16d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32e5b36aa97747d798b79509f3f66dd5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:08.536513 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7ee9de35-2e3d-43da-a110-983c8cd16d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32e5b36aa97747d798b79509f3f66dd5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:08.537280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7ee9de35-2e3d-43da-a110-983c8cd16d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32e5b36aa97747d798b79509f3f66dd5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:08.553299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 849/1678] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:08 2022] DELETE /identity/v3/users/32e5b36aa97747d798b79509f3f66dd5 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:08.558097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e0868-83b9-42a5-b0af-bed60af1c299 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:08.558383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e0868-83b9-42a5-b0af-bed60af1c299 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:08.558663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e0868-83b9-42a5-b0af-bed60af1c299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:08.589613 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d08e0868-83b9-42a5-b0af-bed60af1c299 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:08.590999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d08e0868-83b9-42a5-b0af-bed60af1c299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:08.643550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da278f-4284-4c91-8c98-3b7b15365136 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:08.794739 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:08.799521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 850/1679] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:28:08.816318 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2017a-e618-4dd6-b952-bbbd46eb76ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:08.816943 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2017a-e618-4dd6-b952-bbbd46eb76ad admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:08.817221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2017a-e618-4dd6-b952-bbbd46eb76ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:08.817492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2017a-e618-4dd6-b952-bbbd46eb76ad admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:08.818133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2017a-e618-4dd6-b952-bbbd46eb76ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:08.828790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2017a-e618-4dd6-b952-bbbd46eb76ad admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:08.870676 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 830/1680] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:08 2022] POST /identity/v3/users => generated 286 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:08.874548 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e0868-83b9-42a5-b0af-bed60af1c299 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:08.877287 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d08e0868-83b9-42a5-b0af-bed60af1c299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:08.893243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e0868-83b9-42a5-b0af-bed60af1c299 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:08.902721 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fc5f3-fe5a-461d-ba1e-be8959f0486e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:08.903514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fc5f3-fe5a-461d-ba1e-be8959f0486e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:08.903869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fc5f3-fe5a-461d-ba1e-be8959f0486e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:08.904418 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fc5f3-fe5a-461d-ba1e-be8959f0486e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:08.905080 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fc5f3-fe5a-461d-ba1e-be8959f0486e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:08.908000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fc5f3-fe5a-461d-ba1e-be8959f0486e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:08.934579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2017a-e618-4dd6-b952-bbbd46eb76ad admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:08.938762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d2e2017a-e618-4dd6-b952-bbbd46eb76ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:08.952230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 851/1681] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:08.956439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fc5f3-fe5a-461d-ba1e-be8959f0486e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:08.996763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fc5f3-fe5a-461d-ba1e-be8959f0486e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:09.003560 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2cb9a-3b96-448e-8ef1-f627ee9f0000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:09.004631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2cb9a-3b96-448e-8ef1-f627ee9f0000 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:09.004923 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2cb9a-3b96-448e-8ef1-f627ee9f0000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:09.005204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2cb9a-3b96-448e-8ef1-f627ee9f0000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:09.005935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2cb9a-3b96-448e-8ef1-f627ee9f0000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:09.007790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2cb9a-3b96-448e-8ef1-f627ee9f0000 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:09.022496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 852/1682] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:09.023590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2cb9a-3b96-448e-8ef1-f627ee9f0000 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:09.051379 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2cb9a-3b96-448e-8ef1-f627ee9f0000 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:09.065234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303676fd-f4d4-4966-91cc-c9e7effd9a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:09.067871 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303676fd-f4d4-4966-91cc-c9e7effd9a4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:09.068159 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303676fd-f4d4-4966-91cc-c9e7effd9a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:09.068723 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303676fd-f4d4-4966-91cc-c9e7effd9a4b admin admin] PATH_INFO: `/v3/projects/61cfa2bf1c22469ca56f2824ec26c100/users/ef23ec71f776475d8d0a9812d6a5517e/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:09.074067 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:09.093383 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2017a-e618-4dd6-b952-bbbd46eb76ad admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:09.099498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303676fd-f4d4-4966-91cc-c9e7effd9a4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=61cfa2bf1c22469ca56f2824ec26c100, user_id=ef23ec71f776475d8d0a9812d6a5517e, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:09.102414 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303676fd-f4d4-4966-91cc-c9e7effd9a4b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:09.103321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b77ca6-b035-4bcd-b000-149c0e644c92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:09.104180 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b77ca6-b035-4bcd-b000-149c0e644c92 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:09.108403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b77ca6-b035-4bcd-b000-149c0e644c92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:09.108690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b77ca6-b035-4bcd-b000-149c0e644c92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:09.182076 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 853/1683] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:09 2022] PUT /identity/v3/projects/61cfa2bf1c22469ca56f2824ec26c100/users/ef23ec71f776475d8d0a9812d6a5517e/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:09.185116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b77ca6-b035-4bcd-b000-149c0e644c92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:09.187039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b04aa9-d2e6-4723-a9dd-88c1566e8e78 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:09.187329 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b04aa9-d2e6-4723-a9dd-88c1566e8e78 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:09.188031 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b77ca6-b035-4bcd-b000-149c0e644c92 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:09.188586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b04aa9-d2e6-4723-a9dd-88c1566e8e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:09.227455 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 831/1684] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:09.260024 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-11b04aa9-d2e6-4723-a9dd-88c1566e8e78 None None] MFA Rules not processed for user `ef23ec71f776475d8d0a9812d6a5517e`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:09.261324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-11b04aa9-d2e6-4723-a9dd-88c1566e8e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:09.396144 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 854/1685] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:28:09.397361 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b77ca6-b035-4bcd-b000-149c0e644c92 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:09.414261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b77ca6-b035-4bcd-b000-149c0e644c92 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:09.605444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc974c4-39d2-4b3d-9353-ae142e1d98ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:09.606079 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc974c4-39d2-4b3d-9353-ae142e1d98ae admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:09.606361 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc974c4-39d2-4b3d-9353-ae142e1d98ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:09.607885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc974c4-39d2-4b3d-9353-ae142e1d98ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:09.610866 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc974c4-39d2-4b3d-9353-ae142e1d98ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:09.615776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc974c4-39d2-4b3d-9353-ae142e1d98ae admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:09.679376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 832/1686] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:09 2022] POST /identity/v3/projects => generated 371 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:09.683276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b04aa9-d2e6-4723-a9dd-88c1566e8e78 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:09.721333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b04aa9-d2e6-4723-a9dd-88c1566e8e78 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:09.734724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31976f-5283-4f4c-9a47-a4006f1a4bce None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:09.735180 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31976f-5283-4f4c-9a47-a4006f1a4bce None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:09.735631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31976f-5283-4f4c-9a47-a4006f1a4bce None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:09.739488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 833/1687] 10.222.0.27 () {54 vars in 921 bytes} [Mon Sep 12 23:28:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:28:09.789051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e263f-82b4-475a-a929-693c2c671eb7 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:09.789341 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e263f-82b4-475a-a929-693c2c671eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:09.789624 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e263f-82b4-475a-a929-693c2c671eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:09.818227 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-ee6e263f-82b4-475a-a929-693c2c671eb7 None None] MFA Rules not processed for user `2b3b0a0317b14e29a32eb6ba6b89c2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:09.820147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ee6e263f-82b4-475a-a929-693c2c671eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:09.874195 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd615fde-eca4-42bf-b9bf-0e030a00e249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:09.877358 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd615fde-eca4-42bf-b9bf-0e030a00e249 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:09.877741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd615fde-eca4-42bf-b9bf-0e030a00e249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:09.878099 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd615fde-eca4-42bf-b9bf-0e030a00e249 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:09.878848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd615fde-eca4-42bf-b9bf-0e030a00e249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:09.883048 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd615fde-eca4-42bf-b9bf-0e030a00e249 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:09.951409 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 855/1688] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:09 2022] POST /identity/v3/users => generated 292 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:09.954546 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd615fde-eca4-42bf-b9bf-0e030a00e249 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:09.985812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd615fde-eca4-42bf-b9bf-0e030a00e249 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:10.001774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbf3cd-bf7c-4dd4-b8d1-f6ca50f78312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:10.011275 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf3cd-bf7c-4dd4-b8d1-f6ca50f78312 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:10.011680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf3cd-bf7c-4dd4-b8d1-f6ca50f78312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:10.011970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf3cd-bf7c-4dd4-b8d1-f6ca50f78312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:10.012674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbf3cd-bf7c-4dd4-b8d1-f6ca50f78312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:10.013689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 834/1689] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Sep 12 23:28:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:10.016112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbf3cd-bf7c-4dd4-b8d1-f6ca50f78312 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:10.046811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e263f-82b4-475a-a929-693c2c671eb7 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:10.048296 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ee6e263f-82b4-475a-a929-693c2c671eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:10.065140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 856/1690] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:10.066692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbf3cd-bf7c-4dd4-b8d1-f6ca50f78312 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:10.083806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e263f-82b4-475a-a929-693c2c671eb7 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:10.103805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef857c-db78-46b3-9e2a-0928157619ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b3b0a0317b14e29a32eb6ba6b89c2aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:10.104432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef857c-db78-46b3-9e2a-0928157619ce service placement] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:10.104708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef857c-db78-46b3-9e2a-0928157619ce service placement] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:10.104980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef857c-db78-46b3-9e2a-0928157619ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:10.106325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbf3cd-bf7c-4dd4-b8d1-f6ca50f78312 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:10.124590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61702d-882f-4133-8f18-33660262215f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:10.125283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61702d-882f-4133-8f18-33660262215f admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:10.125569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61702d-882f-4133-8f18-33660262215f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:10.125849 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61702d-882f-4133-8f18-33660262215f admin admin] PATH_INFO: `/v3/projects/b7337005f99b48a5896a97ce1bd20fa5/users/05d3673e22e049c9af243cdfa6a6e681/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:10.128037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:10.139804 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef857c-db78-46b3-9e2a-0928157619ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:10.147839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef857c-db78-46b3-9e2a-0928157619ce service placement] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:10.185064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 835/1691] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:10.188140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61702d-882f-4133-8f18-33660262215f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7337005f99b48a5896a97ce1bd20fa5, user_id=05d3673e22e049c9af243cdfa6a6e681, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:10.193258 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61702d-882f-4133-8f18-33660262215f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:10.267187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 857/1692] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:10 2022] PUT /identity/v3/projects/b7337005f99b48a5896a97ce1bd20fa5/users/05d3673e22e049c9af243cdfa6a6e681/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:10.268962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef857c-db78-46b3-9e2a-0928157619ce service placement] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:10.301019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef857c-db78-46b3-9e2a-0928157619ce service placement] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:10.445620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451c1fd-acc4-4a7c-b54b-d5a286e0b5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:10.447203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451c1fd-acc4-4a7c-b54b-d5a286e0b5b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:10.447808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451c1fd-acc4-4a7c-b54b-d5a286e0b5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:10.448285 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451c1fd-acc4-4a7c-b54b-d5a286e0b5b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:10.449204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451c1fd-acc4-4a7c-b54b-d5a286e0b5b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:10.451332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451c1fd-acc4-4a7c-b54b-d5a286e0b5b6 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:10.467224 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 836/1693] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:10.467504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61702d-882f-4133-8f18-33660262215f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:10.479508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61702d-882f-4133-8f18-33660262215f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:10.490586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699ff41-3d6c-434d-8bc0-dfc81771f528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:10.491754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699ff41-3d6c-434d-8bc0-dfc81771f528 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:10.492286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699ff41-3d6c-434d-8bc0-dfc81771f528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:10.492840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699ff41-3d6c-434d-8bc0-dfc81771f528 admin admin] PATH_INFO: `/v3/projects/b7337005f99b48a5896a97ce1bd20fa5/users/05d3673e22e049c9af243cdfa6a6e681/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:10.496550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699ff41-3d6c-434d-8bc0-dfc81771f528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7337005f99b48a5896a97ce1bd20fa5, user_id=05d3673e22e049c9af243cdfa6a6e681, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:10.501040 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699ff41-3d6c-434d-8bc0-dfc81771f528 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:10.521761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 858/1694] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:10 2022] PUT /identity/v3/projects/b7337005f99b48a5896a97ce1bd20fa5/users/05d3673e22e049c9af243cdfa6a6e681/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:10.532991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658faee-fcde-47a5-95b2-b033d9c3139e None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:10.533512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658faee-fcde-47a5-95b2-b033d9c3139e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:10.533892 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658faee-fcde-47a5-95b2-b033d9c3139e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:10.569748 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-4658faee-fcde-47a5-95b2-b033d9c3139e None None] MFA Rules not processed for user `05d3673e22e049c9af243cdfa6a6e681`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:10.572052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4658faee-fcde-47a5-95b2-b033d9c3139e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:10.699024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 837/1695] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:10.715107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699ff41-3d6c-434d-8bc0-dfc81771f528 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:10.717255 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e699ff41-3d6c-434d-8bc0-dfc81771f528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:10.741397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:10.784491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75610d9e-ab74-4e1e-8606-79afd28143eb None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:10.784782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75610d9e-ab74-4e1e-8606-79afd28143eb None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:10.785063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75610d9e-ab74-4e1e-8606-79afd28143eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:10.847781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-75610d9e-ab74-4e1e-8606-79afd28143eb None None] MFA Rules not processed for user `7f56dde21c754cfe8253ec5c1b7b90d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:10.850727 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-75610d9e-ab74-4e1e-8606-79afd28143eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:10.881228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699ff41-3d6c-434d-8bc0-dfc81771f528 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:10.893364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab69f99-8262-49e0-8aff-089247b92a35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:10.894005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab69f99-8262-49e0-8aff-089247b92a35 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:10.894285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab69f99-8262-49e0-8aff-089247b92a35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:10.894563 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab69f99-8262-49e0-8aff-089247b92a35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:10.927763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab69f99-8262-49e0-8aff-089247b92a35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:10.929301 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab69f99-8262-49e0-8aff-089247b92a35 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:10.963001 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 859/1696] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:11.060390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 838/1697] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:11.080199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab69f99-8262-49e0-8aff-089247b92a35 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:11.081714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bab69f99-8262-49e0-8aff-089247b92a35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:11.291759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab69f99-8262-49e0-8aff-089247b92a35 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:11.308454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b1d95-0f3e-4727-a87c-40a7a354b7c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:11.309250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b1d95-0f3e-4727-a87c-40a7a354b7c1 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:11.309537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b1d95-0f3e-4727-a87c-40a7a354b7c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:11.309818 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b1d95-0f3e-4727-a87c-40a7a354b7c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:11.333850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b1d95-0f3e-4727-a87c-40a7a354b7c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:11.336061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b1d95-0f3e-4727-a87c-40a7a354b7c1 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:11.392058 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 860/1698] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:11.732774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75610d9e-ab74-4e1e-8606-79afd28143eb None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:11.734332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-75610d9e-ab74-4e1e-8606-79afd28143eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:11.741818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:11.850229 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b1d95-0f3e-4727-a87c-40a7a354b7c1 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:11.890897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75610d9e-ab74-4e1e-8606-79afd28143eb None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:11.903350 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad856d9-e419-4568-aa8a-b0081da0fc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:11.905432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad856d9-e419-4568-aa8a-b0081da0fc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:11.905962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad856d9-e419-4568-aa8a-b0081da0fc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:11.906552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad856d9-e419-4568-aa8a-b0081da0fc96 admin admin] PATH_INFO: `/v3/projects/c2bcb31c2c0646ff80d50c51c842db89` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:11.910042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad856d9-e419-4568-aa8a-b0081da0fc96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2bcb31c2c0646ff80d50c51c842db89)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:11.914428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad856d9-e419-4568-aa8a-b0081da0fc96 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:12.098380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b1d95-0f3e-4727-a87c-40a7a354b7c1 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:12.101606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4ad856d9-e419-4568-aa8a-b0081da0fc96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2bcb31c2c0646ff80d50c51c842db89', 'request_id': 'req-4ad856d9-e419-4568-aa8a-b0081da0fc96'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:12.116266 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4ad856d9-e419-4568-aa8a-b0081da0fc96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2bcb31c2c0646ff80d50c51c842db89', 'request_id': 'req-4ad856d9-e419-4568-aa8a-b0081da0fc96'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:12.130129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4ad856d9-e419-4568-aa8a-b0081da0fc96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2bcb31c2c0646ff80d50c51c842db89', 'request_id': 'req-4ad856d9-e419-4568-aa8a-b0081da0fc96'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:12.168931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4ad856d9-e419-4568-aa8a-b0081da0fc96 admin admin] The token cache is being invalidate because project c2bcb31c2c0646ff80d50c51c842db89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:12.169395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4ad856d9-e419-4568-aa8a-b0081da0fc96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:12.171861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4ad856d9-e419-4568-aa8a-b0081da0fc96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:12.185269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 839/1699] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:11 2022] DELETE /identity/v3/projects/c2bcb31c2c0646ff80d50c51c842db89 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:12.194083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97374ec-8aeb-471f-b2f2-7bc82c394783 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:12.194577 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97374ec-8aeb-471f-b2f2-7bc82c394783 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:12.195290 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97374ec-8aeb-471f-b2f2-7bc82c394783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:12.213390 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:12.241217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38348cf6-5c57-4f07-bbd8-ed83af34931b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:12.244058 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38348cf6-5c57-4f07-bbd8-ed83af34931b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:12.248297 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38348cf6-5c57-4f07-bbd8-ed83af34931b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:12.248672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38348cf6-5c57-4f07-bbd8-ed83af34931b admin admin] PATH_INFO: `/v3/users/ef23ec71f776475d8d0a9812d6a5517e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:12.256495 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38348cf6-5c57-4f07-bbd8-ed83af34931b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef23ec71f776475d8d0a9812d6a5517e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:12.260444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38348cf6-5c57-4f07-bbd8-ed83af34931b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:12.286094 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-a97374ec-8aeb-471f-b2f2-7bc82c394783 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:12.287400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a97374ec-8aeb-471f-b2f2-7bc82c394783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:12.299837 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 840/1700] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:28:12.302279 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97374ec-8aeb-471f-b2f2-7bc82c394783 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:12.303750 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a97374ec-8aeb-471f-b2f2-7bc82c394783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:12.385142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-38348cf6-5c57-4f07-bbd8-ed83af34931b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef23ec71f776475d8d0a9812d6a5517e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:12.406171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97374ec-8aeb-471f-b2f2-7bc82c394783 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:12.413053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27b5dc-534c-4206-a0d9-6b8ed8dc3fef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:12.414944 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27b5dc-534c-4206-a0d9-6b8ed8dc3fef service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:12.415639 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27b5dc-534c-4206-a0d9-6b8ed8dc3fef service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:12.416060 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27b5dc-534c-4206-a0d9-6b8ed8dc3fef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:12.418745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9e27b5dc-534c-4206-a0d9-6b8ed8dc3fef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:12.422517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-38348cf6-5c57-4f07-bbd8-ed83af34931b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef23ec71f776475d8d0a9812d6a5517e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:12.450198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-38348cf6-5c57-4f07-bbd8-ed83af34931b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef23ec71f776475d8d0a9812d6a5517e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:12.469516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-38348cf6-5c57-4f07-bbd8-ed83af34931b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef23ec71f776475d8d0a9812d6a5517e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:12.470142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-38348cf6-5c57-4f07-bbd8-ed83af34931b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef23ec71f776475d8d0a9812d6a5517e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:12.487213 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 861/1701] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:11 2022] DELETE /identity/v3/users/ef23ec71f776475d8d0a9812d6a5517e => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:12.488857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e9032-4b09-4899-a253-92f2997778d4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:12.490559 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e9032-4b09-4899-a253-92f2997778d4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:12.490994 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e9032-4b09-4899-a253-92f2997778d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:12.527487 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:12.545750 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-965e9032-4b09-4899-a253-92f2997778d4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:12.547032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-965e9032-4b09-4899-a253-92f2997778d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:12.578462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e27b5dc-534c-4206-a0d9-6b8ed8dc3fef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:12.581845 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e27b5dc-534c-4206-a0d9-6b8ed8dc3fef service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:12.604831 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 841/1702] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:12.606320 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e1dbc-321b-4424-b863-2c06e91eef89 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:12.606610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e1dbc-321b-4424-b863-2c06e91eef89 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:12.608088 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e1dbc-321b-4424-b863-2c06e91eef89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:12.678096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 862/1703] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:12.683016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e9032-4b09-4899-a253-92f2997778d4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:12.702419 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e9032-4b09-4899-a253-92f2997778d4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:12.711279 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c22e1dbc-321b-4424-b863-2c06e91eef89 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:12.716382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7407c9-d709-4c1f-8a79-40b3fecfacdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:12.716871 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c22e1dbc-321b-4424-b863-2c06e91eef89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:12.717007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7407c9-d709-4c1f-8a79-40b3fecfacdc admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:12.717291 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7407c9-d709-4c1f-8a79-40b3fecfacdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:12.717567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7407c9-d709-4c1f-8a79-40b3fecfacdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:12.718233 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7407c9-d709-4c1f-8a79-40b3fecfacdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:28:12.718542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7407c9-d709-4c1f-8a79-40b3fecfacdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:12.720193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7407c9-d709-4c1f-8a79-40b3fecfacdc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:12.739650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 863/1704] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:28:12.744055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7407c9-d709-4c1f-8a79-40b3fecfacdc admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:12.762599 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 842/1705] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:12.763962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f7078-dfb1-49b3-bedc-a914ba26eece None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:12.764250 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f7078-dfb1-49b3-bedc-a914ba26eece None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:12.764534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f7078-dfb1-49b3-bedc-a914ba26eece None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:12.766034 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 843/1706] 10.222.0.27 () {54 vars in 921 bytes} [Mon Sep 12 23:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:28:12.769685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7407c9-d709-4c1f-8a79-40b3fecfacdc admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:12.785419 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bb506-ce0e-4619-be6f-1536e7f59c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:12.786033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bb506-ce0e-4619-be6f-1536e7f59c86 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:12.786317 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bb506-ce0e-4619-be6f-1536e7f59c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:12.786594 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bb506-ce0e-4619-be6f-1536e7f59c86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:12.787239 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bb506-ce0e-4619-be6f-1536e7f59c86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:12.795321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bb506-ce0e-4619-be6f-1536e7f59c86 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:12.842499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f7078-dfb1-49b3-bedc-a914ba26eece None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:12.846442 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-f32f7078-dfb1-49b3-bedc-a914ba26eece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:12.859646 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 864/1707] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:12 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:12.864161 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bb506-ce0e-4619-be6f-1536e7f59c86 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:13.004527 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:13.051986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f7078-dfb1-49b3-bedc-a914ba26eece None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:13.098432 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bb506-ce0e-4619-be6f-1536e7f59c86 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:13.125684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d454fb-15a9-42ee-8e79-d13d59753956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:13.126410 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d454fb-15a9-42ee-8e79-d13d59753956 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:13.126618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d454fb-15a9-42ee-8e79-d13d59753956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:13.128103 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d454fb-15a9-42ee-8e79-d13d59753956 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:13.133510 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d454fb-15a9-42ee-8e79-d13d59753956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:13.136724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d454fb-15a9-42ee-8e79-d13d59753956 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:13.206165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 865/1708] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:12 2022] POST /identity/v3/users => generated 287 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:13.207861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d5db2-15e8-4342-affd-ca1ef6c0880c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:13.209736 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d5db2-15e8-4342-affd-ca1ef6c0880c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:13.210344 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d5db2-15e8-4342-affd-ca1ef6c0880c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:13.235329 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702d760-00c5-43ce-8c1b-86b05faf9259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:13.237759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702d760-00c5-43ce-8c1b-86b05faf9259 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:13.240263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702d760-00c5-43ce-8c1b-86b05faf9259 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:13.245957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702d760-00c5-43ce-8c1b-86b05faf9259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:13.273713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7b8d5db2-15e8-4342-affd-ca1ef6c0880c None None] MFA Rules not processed for user `2b3b0a0317b14e29a32eb6ba6b89c2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:13.275225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702d760-00c5-43ce-8c1b-86b05faf9259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:13.276461 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702d760-00c5-43ce-8c1b-86b05faf9259 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:13.276865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7b8d5db2-15e8-4342-affd-ca1ef6c0880c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:13.296053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 844/1709] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:28:13.302605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702d760-00c5-43ce-8c1b-86b05faf9259 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:13.334961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702d760-00c5-43ce-8c1b-86b05faf9259 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:13.348883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d604fa-9a9f-4a6e-9cd6-f9c011df9ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:13.349900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d604fa-9a9f-4a6e-9cd6-f9c011df9ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:13.350577 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d604fa-9a9f-4a6e-9cd6-f9c011df9ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:13.351041 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d604fa-9a9f-4a6e-9cd6-f9c011df9ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:13.353173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d604fa-9a9f-4a6e-9cd6-f9c011df9ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:13.357930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d604fa-9a9f-4a6e-9cd6-f9c011df9ce5 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:13.381777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 845/1710] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:13.383772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d604fa-9a9f-4a6e-9cd6-f9c011df9ce5 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:13.387930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b3d604fa-9a9f-4a6e-9cd6-f9c011df9ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:13.444806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d604fa-9a9f-4a6e-9cd6-f9c011df9ce5 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:13.464919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc687ff-7bad-4806-9200-51270430b6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:13.466962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc687ff-7bad-4806-9200-51270430b6f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:13.469167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc687ff-7bad-4806-9200-51270430b6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:13.470754 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc687ff-7bad-4806-9200-51270430b6f8 admin admin] PATH_INFO: `/v3/users/a29f8dda51c44e08a46fabe3fe9f86ce` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:13.473393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc687ff-7bad-4806-9200-51270430b6f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a29f8dda51c44e08a46fabe3fe9f86ce)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:13.478211 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc687ff-7bad-4806-9200-51270430b6f8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:13.480828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 866/1711] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Sep 12 23:28:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:13.482016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d5db2-15e8-4342-affd-ca1ef6c0880c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:13.508378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d5db2-15e8-4342-affd-ca1ef6c0880c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:13.526514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959e555-cb74-4cf1-825c-f01e56b32da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:13.531261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959e555-cb74-4cf1-825c-f01e56b32da0 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:13.531767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959e555-cb74-4cf1-825c-f01e56b32da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:13.532359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959e555-cb74-4cf1-825c-f01e56b32da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:13.533232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7959e555-cb74-4cf1-825c-f01e56b32da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:13.536586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7959e555-cb74-4cf1-825c-f01e56b32da0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:13.572772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 867/1712] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:13.574630 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959e555-cb74-4cf1-825c-f01e56b32da0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:13.593906 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959e555-cb74-4cf1-825c-f01e56b32da0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:13.596206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7bc687ff-7bad-4806-9200-51270430b6f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a29f8dda51c44e08a46fabe3fe9f86ce'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:13.607168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511e485-d1fa-44c0-a13e-682518b88b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:13.607897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511e485-d1fa-44c0-a13e-682518b88b9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:13.608153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511e485-d1fa-44c0-a13e-682518b88b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:13.608427 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511e485-d1fa-44c0-a13e-682518b88b9e admin admin] PATH_INFO: `/v3/projects/148b03d0d93f428ca93a67e409078a9c/users/2179bb8a78874947ac676c2f7f9329fd/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:13.633595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7bc687ff-7bad-4806-9200-51270430b6f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a29f8dda51c44e08a46fabe3fe9f86ce'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:13.648724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7bc687ff-7bad-4806-9200-51270430b6f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a29f8dda51c44e08a46fabe3fe9f86ce'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:13.649746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7bc687ff-7bad-4806-9200-51270430b6f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a29f8dda51c44e08a46fabe3fe9f86ce'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:13.654828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:13.655300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511e485-d1fa-44c0-a13e-682518b88b9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=148b03d0d93f428ca93a67e409078a9c, user_id=2179bb8a78874947ac676c2f7f9329fd, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:13.659929 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511e485-d1fa-44c0-a13e-682518b88b9e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:13.669058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7bc687ff-7bad-4806-9200-51270430b6f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a29f8dda51c44e08a46fabe3fe9f86ce'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:13.686930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 846/1713] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:13 2022] DELETE /identity/v3/users/a29f8dda51c44e08a46fabe3fe9f86ce => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:13.690373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc687ff-7bad-4806-9200-51270430b6f8 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:13.695217 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7bc687ff-7bad-4806-9200-51270430b6f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:13.697380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 868/1714] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:13 2022] PUT /identity/v3/projects/148b03d0d93f428ca93a67e409078a9c/users/2179bb8a78874947ac676c2f7f9329fd/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:13.709761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d1720-2839-4c63-bfff-e7a08dcba53b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:13.710174 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d1720-2839-4c63-bfff-e7a08dcba53b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:13.710587 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d1720-2839-4c63-bfff-e7a08dcba53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:13.775031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-440d1720-2839-4c63-bfff-e7a08dcba53b None None] MFA Rules not processed for user `2179bb8a78874947ac676c2f7f9329fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:13.777764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-440d1720-2839-4c63-bfff-e7a08dcba53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:13.852043 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc687ff-7bad-4806-9200-51270430b6f8 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:13.863482 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb152cc-df74-4ff9-a0da-54fe356caced service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b3b0a0317b14e29a32eb6ba6b89c2aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:13.864427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb152cc-df74-4ff9-a0da-54fe356caced service placement] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:13.867226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb152cc-df74-4ff9-a0da-54fe356caced service placement] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:13.867621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb152cc-df74-4ff9-a0da-54fe356caced service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:13.873603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7eb152cc-df74-4ff9-a0da-54fe356caced service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:13.900308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb152cc-df74-4ff9-a0da-54fe356caced service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:13.904316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb152cc-df74-4ff9-a0da-54fe356caced service placement] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:13.943492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 847/1715] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:13.955758 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 869/1716] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:13.957488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb152cc-df74-4ff9-a0da-54fe356caced service placement] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:13.959429 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7eb152cc-df74-4ff9-a0da-54fe356caced service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:14.058078 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb152cc-df74-4ff9-a0da-54fe356caced service placement] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:14.063306 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443ed7d-22f1-46b4-8364-f418328a4864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:14.064043 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443ed7d-22f1-46b4-8364-f418328a4864 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:14.065374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443ed7d-22f1-46b4-8364-f418328a4864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:14.065705 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443ed7d-22f1-46b4-8364-f418328a4864 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:14.066377 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443ed7d-22f1-46b4-8364-f418328a4864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:14.068598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443ed7d-22f1-46b4-8364-f418328a4864 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:14.108525 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 848/1717] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:13 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:28:14.112759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d1720-2839-4c63-bfff-e7a08dcba53b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:14.131163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d1720-2839-4c63-bfff-e7a08dcba53b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:14.145266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:14.262835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fadcfb-ce25-45b2-9192-e803404b004c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:14.263763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fadcfb-ce25-45b2-9192-e803404b004c admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:14.264194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fadcfb-ce25-45b2-9192-e803404b004c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:14.264762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fadcfb-ce25-45b2-9192-e803404b004c admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:14.265683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fadcfb-ce25-45b2-9192-e803404b004c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:14.268941 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fadcfb-ce25-45b2-9192-e803404b004c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:14.306409 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 870/1718] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:14 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:14.308949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443ed7d-22f1-46b4-8364-f418328a4864 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:14.334045 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee14a3-b744-413c-a5bb-8e0f11fcb507 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:14.334343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee14a3-b744-413c-a5bb-8e0f11fcb507 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:14.334626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee14a3-b744-413c-a5bb-8e0f11fcb507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:14.337817 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443ed7d-22f1-46b4-8364-f418328a4864 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:14.350419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c201d0-45ad-4105-be3e-2fb96dd6f1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:14.354127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c201d0-45ad-4105-be3e-2fb96dd6f1de admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:14.354514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c201d0-45ad-4105-be3e-2fb96dd6f1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:14.354885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c201d0-45ad-4105-be3e-2fb96dd6f1de admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:14.355675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c201d0-45ad-4105-be3e-2fb96dd6f1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:14.356127 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c6ee14a3-b744-413c-a5bb-8e0f11fcb507 None None] MFA Rules not processed for user `2749a61b3dd04d89a5c2e4235aa96934`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:14.357980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c6ee14a3-b744-413c-a5bb-8e0f11fcb507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:14.360073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c201d0-45ad-4105-be3e-2fb96dd6f1de admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:14.390193 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 849/1719] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:14.394775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c201d0-45ad-4105-be3e-2fb96dd6f1de admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:14.413129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c201d0-45ad-4105-be3e-2fb96dd6f1de admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:14.424687 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949da1d-8ceb-447e-abdd-dbaa3befd61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:14.425485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949da1d-8ceb-447e-abdd-dbaa3befd61e admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:14.425776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949da1d-8ceb-447e-abdd-dbaa3befd61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:14.426110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949da1d-8ceb-447e-abdd-dbaa3befd61e admin admin] PATH_INFO: `/v3/projects/f67b152f8283494eaffcb4fb056758ad/users/0c6f00d24905449c887b1a2f8149b4ff/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:14.447037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 871/1720] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:14.473570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949da1d-8ceb-447e-abdd-dbaa3befd61e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f67b152f8283494eaffcb4fb056758ad, user_id=0c6f00d24905449c887b1a2f8149b4ff, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:14.476335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949da1d-8ceb-447e-abdd-dbaa3befd61e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:14.523745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 850/1721] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:14 2022] PUT /identity/v3/projects/f67b152f8283494eaffcb4fb056758ad/users/0c6f00d24905449c887b1a2f8149b4ff/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:28:14.527776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee14a3-b744-413c-a5bb-8e0f11fcb507 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:14.552307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee14a3-b744-413c-a5bb-8e0f11fcb507 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:14.802080 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066defe8-3e28-44e0-83f5-c78cb9dc4322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:14.802734 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066defe8-3e28-44e0-83f5-c78cb9dc4322 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:14.803020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066defe8-3e28-44e0-83f5-c78cb9dc4322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:14.803300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066defe8-3e28-44e0-83f5-c78cb9dc4322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:14.803995 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066defe8-3e28-44e0-83f5-c78cb9dc4322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:14.805915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066defe8-3e28-44e0-83f5-c78cb9dc4322 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:14.833457 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 872/1722] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:14.835741 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949da1d-8ceb-447e-abdd-dbaa3befd61e admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:14.850669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949da1d-8ceb-447e-abdd-dbaa3befd61e admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:14.861324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12643d1f-fe46-4b4c-9210-f462169e33d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:14.862785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12643d1f-fe46-4b4c-9210-f462169e33d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:14.863085 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12643d1f-fe46-4b4c-9210-f462169e33d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:14.863736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12643d1f-fe46-4b4c-9210-f462169e33d5 admin admin] PATH_INFO: `/v3/projects/f67b152f8283494eaffcb4fb056758ad/users/0c6f00d24905449c887b1a2f8149b4ff/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:14.870366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12643d1f-fe46-4b4c-9210-f462169e33d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f67b152f8283494eaffcb4fb056758ad, user_id=0c6f00d24905449c887b1a2f8149b4ff, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:14.877830 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12643d1f-fe46-4b4c-9210-f462169e33d5 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:14.906198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 851/1723] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:14 2022] PUT /identity/v3/projects/f67b152f8283494eaffcb4fb056758ad/users/0c6f00d24905449c887b1a2f8149b4ff/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:14.916519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba6656-6b8c-4de5-ae0f-99e36e378a18 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:14.917026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba6656-6b8c-4de5-ae0f-99e36e378a18 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:14.917919 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba6656-6b8c-4de5-ae0f-99e36e378a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:14.977838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-35ba6656-6b8c-4de5-ae0f-99e36e378a18 None None] MFA Rules not processed for user `0c6f00d24905449c887b1a2f8149b4ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:14.979544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-35ba6656-6b8c-4de5-ae0f-99e36e378a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:15.005450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12643d1f-fe46-4b4c-9210-f462169e33d5 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:15.007608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-12643d1f-fe46-4b4c-9210-f462169e33d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:15.057036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:15.093766 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:15.192619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 873/1724] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:15.202242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12643d1f-fe46-4b4c-9210-f462169e33d5 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:15.211333 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:15.212240 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:15.213008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:15.213467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] PATH_INFO: `/v3/projects/3055070ceb48447e9c6e05d102ad748f` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:15.216522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba6656-6b8c-4de5-ae0f-99e36e378a18 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:15.218389 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-35ba6656-6b8c-4de5-ae0f-99e36e378a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:15.226542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3055070ceb48447e9c6e05d102ad748f)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:15.232180 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:15.348782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba6656-6b8c-4de5-ae0f-99e36e378a18 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:15.362351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3055070ceb48447e9c6e05d102ad748f', 'request_id': 'req-36500fed-0deb-4071-b415-fe92582e687d'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.379447 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3055070ceb48447e9c6e05d102ad748f', 'request_id': 'req-36500fed-0deb-4071-b415-fe92582e687d'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.411051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3055070ceb48447e9c6e05d102ad748f', 'request_id': 'req-36500fed-0deb-4071-b415-fe92582e687d'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.435852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] The token cache is being invalidate because project 3055070ceb48447e9c6e05d102ad748f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:15.436143 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.436995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.444253 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 852/1725] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:15 2022] DELETE /identity/v3/projects/3055070ceb48447e9c6e05d102ad748f => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:15.445763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:15.447996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:15.450189 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ca059-7956-45d7-94db-dc7ed061c585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:15.450902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ca059-7956-45d7-94db-dc7ed061c585 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:15.455459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ca059-7956-45d7-94db-dc7ed061c585 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:15.455866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ca059-7956-45d7-94db-dc7ed061c585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:15.465032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5f6ca059-7956-45d7-94db-dc7ed061c585 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:15.631849 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ca059-7956-45d7-94db-dc7ed061c585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:15.634490 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ca059-7956-45d7-94db-dc7ed061c585 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:15.643318 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36500fed-0deb-4071-b415-fe92582e687d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:15.668421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72815e-33a1-4439-b36e-767fca0690f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:15.669991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72815e-33a1-4439-b36e-767fca0690f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:15.670381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72815e-33a1-4439-b36e-767fca0690f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:15.670959 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72815e-33a1-4439-b36e-767fca0690f4 admin admin] PATH_INFO: `/v3/users/04017f4b4b0547719538ae7befa0ea17` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:15.672868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72815e-33a1-4439-b36e-767fca0690f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04017f4b4b0547719538ae7befa0ea17)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:15.676051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72815e-33a1-4439-b36e-767fca0690f4 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:15.681565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 874/1726] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:15.685184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ca059-7956-45d7-94db-dc7ed061c585 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:15.689926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5f6ca059-7956-45d7-94db-dc7ed061c585 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:15.726221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ca059-7956-45d7-94db-dc7ed061c585 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:15.733761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:15.737101 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:15.737390 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:15.737655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] PATH_INFO: `/v3/projects/61cfa2bf1c22469ca56f2824ec26c100` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:15.739460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] RBAC: Authorizing `identity:delete_project(project_id=61cfa2bf1c22469ca56f2824ec26c100)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:15.743598 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:15.779855 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1d72815e-33a1-4439-b36e-767fca0690f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04017f4b4b0547719538ae7befa0ea17'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.785901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:15.808096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:15.816626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1d72815e-33a1-4439-b36e-767fca0690f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04017f4b4b0547719538ae7befa0ea17'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.829217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61cfa2bf1c22469ca56f2824ec26c100', 'request_id': 'req-f5cfff16-14c5-490d-a438-9d9c1e99736f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.836983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61cfa2bf1c22469ca56f2824ec26c100', 'request_id': 'req-f5cfff16-14c5-490d-a438-9d9c1e99736f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.851268 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61cfa2bf1c22469ca56f2824ec26c100', 'request_id': 'req-f5cfff16-14c5-490d-a438-9d9c1e99736f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.851905 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1d72815e-33a1-4439-b36e-767fca0690f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04017f4b4b0547719538ae7befa0ea17'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.852721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1d72815e-33a1-4439-b36e-767fca0690f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04017f4b4b0547719538ae7befa0ea17'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.865213 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1d72815e-33a1-4439-b36e-767fca0690f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04017f4b4b0547719538ae7befa0ea17'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.869027 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] The token cache is being invalidate because project 61cfa2bf1c22469ca56f2824ec26c100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:15.869325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.870358 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:15.872268 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 875/1727] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:15 2022] DELETE /identity/v3/projects/61cfa2bf1c22469ca56f2824ec26c100 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:15.872836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:15.874336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:15.900073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 853/1728] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:15 2022] DELETE /identity/v3/users/04017f4b4b0547719538ae7befa0ea17 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:15.901276 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72815e-33a1-4439-b36e-767fca0690f4 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:15.903183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1d72815e-33a1-4439-b36e-767fca0690f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:16.001020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfff16-14c5-490d-a438-9d9c1e99736f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:16.052374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72815e-33a1-4439-b36e-767fca0690f4 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:16.067076 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53537596-fc12-45b5-8fa5-b4eafedf5f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:16.067778 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53537596-fc12-45b5-8fa5-b4eafedf5f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:16.068066 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53537596-fc12-45b5-8fa5-b4eafedf5f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:16.068345 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53537596-fc12-45b5-8fa5-b4eafedf5f89 admin admin] PATH_INFO: `/v3/users/05d3673e22e049c9af243cdfa6a6e681` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:16.070943 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53537596-fc12-45b5-8fa5-b4eafedf5f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05d3673e22e049c9af243cdfa6a6e681)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:16.073737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53537596-fc12-45b5-8fa5-b4eafedf5f89 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:16.138181 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da257f-3305-4f7b-b1c7-6983afdb490f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:16.139075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da257f-3305-4f7b-b1c7-6983afdb490f service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:16.139075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da257f-3305-4f7b-b1c7-6983afdb490f service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:16.139350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da257f-3305-4f7b-b1c7-6983afdb490f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:16.145601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-98da257f-3305-4f7b-b1c7-6983afdb490f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:16.214753 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-53537596-fc12-45b5-8fa5-b4eafedf5f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05d3673e22e049c9af243cdfa6a6e681'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:16.257892 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-53537596-fc12-45b5-8fa5-b4eafedf5f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05d3673e22e049c9af243cdfa6a6e681'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:16.281461 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-53537596-fc12-45b5-8fa5-b4eafedf5f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05d3673e22e049c9af243cdfa6a6e681'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:16.283983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-53537596-fc12-45b5-8fa5-b4eafedf5f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05d3673e22e049c9af243cdfa6a6e681'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:16.307665 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-53537596-fc12-45b5-8fa5-b4eafedf5f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05d3673e22e049c9af243cdfa6a6e681'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:16.327659 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 854/1729] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:15 2022] DELETE /identity/v3/users/05d3673e22e049c9af243cdfa6a6e681 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:16.327659 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22f017-66cf-4a6a-880c-860bc3d86cd9 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:16.327659 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22f017-66cf-4a6a-880c-860bc3d86cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:16.327659 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22f017-66cf-4a6a-880c-860bc3d86cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:16.353707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da257f-3305-4f7b-b1c7-6983afdb490f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:16.355235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da257f-3305-4f7b-b1c7-6983afdb490f service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:16.380069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 876/1730] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:16.381626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccd626-971e-4736-b608-8e4de5da9c58 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:16.381941 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccd626-971e-4736-b608-8e4de5da9c58 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:16.382230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccd626-971e-4736-b608-8e4de5da9c58 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:16.384135 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 877/1731] 10.222.0.27 () {54 vars in 912 bytes} [Mon Sep 12 23:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:28:16.390460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73040b-513d-4b27-9fe2-bd8912caa2f0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:16.390784 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73040b-513d-4b27-9fe2-bd8912caa2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:16.391034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73040b-513d-4b27-9fe2-bd8912caa2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:16.415150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-8d22f017-66cf-4a6a-880c-860bc3d86cd9 None None] MFA Rules not processed for user `0248519bf5b54ce3ba3b40f2c9a4c628`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:16.418379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8d22f017-66cf-4a6a-880c-860bc3d86cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:16.515305 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:16.537269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1a73040b-513d-4b27-9fe2-bd8912caa2f0 None None] MFA Rules not processed for user `bc6e63e9fc3447d5a894a777a98fe773`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:16.538742 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1a73040b-513d-4b27-9fe2-bd8912caa2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:16.575493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 855/1732] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:16.575493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22f017-66cf-4a6a-880c-860bc3d86cd9 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:16.575493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8d22f017-66cf-4a6a-880c-860bc3d86cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:16.600124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:16.692344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22f017-66cf-4a6a-880c-860bc3d86cd9 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:16.698290 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:16.698955 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:16.699226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:16.699504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] PATH_INFO: `/v3/projects/370e5966bb6a4eda8567bad550f443f2` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:16.700859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=370e5966bb6a4eda8567bad550f443f2)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:16.705664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:16.791061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 878/1733] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:28:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:16.797812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '370e5966bb6a4eda8567bad550f443f2', 'request_id': 'req-6706109f-2612-4c9b-96b1-03f6c78cf6c2'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:16.805133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be393f0-3765-4107-b600-33e337fa8992 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:16.805409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '370e5966bb6a4eda8567bad550f443f2', 'request_id': 'req-6706109f-2612-4c9b-96b1-03f6c78cf6c2'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:16.807909 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be393f0-3765-4107-b600-33e337fa8992 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:16.810681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be393f0-3765-4107-b600-33e337fa8992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:16.817228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '370e5966bb6a4eda8567bad550f443f2', 'request_id': 'req-6706109f-2612-4c9b-96b1-03f6c78cf6c2'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:16.836193 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] The token cache is being invalidate because project 370e5966bb6a4eda8567bad550f443f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:16.836502 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:16.837199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:16.839428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 856/1734] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:16 2022] DELETE /identity/v3/projects/370e5966bb6a4eda8567bad550f443f2 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:16.846697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:16.848652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:16.909145 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6be393f0-3765-4107-b600-33e337fa8992 None None] MFA Rules not processed for user `0248519bf5b54ce3ba3b40f2c9a4c628`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:16.922459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6be393f0-3765-4107-b600-33e337fa8992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:17.020911 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706109f-2612-4c9b-96b1-03f6c78cf6c2 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:17.025742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd2186-dbcf-48f8-b586-094b0da90641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:17.027058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd2186-dbcf-48f8-b586-094b0da90641 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:17.028183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd2186-dbcf-48f8-b586-094b0da90641 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:17.028851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd2186-dbcf-48f8-b586-094b0da90641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:17.032005 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b3fd2186-dbcf-48f8-b586-094b0da90641 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:17.059758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd2186-dbcf-48f8-b586-094b0da90641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:17.061971 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd2186-dbcf-48f8-b586-094b0da90641 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:17.071441 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 879/1735] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:17.074755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 857/1736] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:17.085229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd2186-dbcf-48f8-b586-094b0da90641 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:17.085811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555bc8c-6db0-4747-bf28-0d9d56e4183a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:17.086105 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555bc8c-6db0-4747-bf28-0d9d56e4183a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:17.086384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555bc8c-6db0-4747-bf28-0d9d56e4183a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:17.089355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b3fd2186-dbcf-48f8-b586-094b0da90641 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:17.109805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:17.123065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0555bc8c-6db0-4747-bf28-0d9d56e4183a None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:17.124489 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0555bc8c-6db0-4747-bf28-0d9d56e4183a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:17.200106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 880/1737] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:17.210795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391acf9-cd44-40f2-be81-6197b9c4cc8c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:17.211115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391acf9-cd44-40f2-be81-6197b9c4cc8c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:17.211461 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391acf9-cd44-40f2-be81-6197b9c4cc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:17.231186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd2186-dbcf-48f8-b586-094b0da90641 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:17.240941 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b391acf9-cd44-40f2-be81-6197b9c4cc8c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:17.242891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b391acf9-cd44-40f2-be81-6197b9c4cc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:17.254772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0dc7c-f89d-4e44-80df-26c3ca070a85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:17.255752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0dc7c-f89d-4e44-80df-26c3ca070a85 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:17.257202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 881/1738] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:17.260064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0dc7c-f89d-4e44-80df-26c3ca070a85 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:17.260875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0dc7c-f89d-4e44-80df-26c3ca070a85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:17.263099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e5b0dc7c-f89d-4e44-80df-26c3ca070a85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:17.273641 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391acf9-cd44-40f2-be81-6197b9c4cc8c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:17.296523 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391acf9-cd44-40f2-be81-6197b9c4cc8c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:17.316923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0dc7c-f89d-4e44-80df-26c3ca070a85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:17.320072 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891f260-8eb1-433c-8e17-739279f616db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:17.320747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891f260-8eb1-433c-8e17-739279f616db admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:17.320982 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891f260-8eb1-433c-8e17-739279f616db admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:17.321253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891f260-8eb1-433c-8e17-739279f616db admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:17.321917 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891f260-8eb1-433c-8e17-739279f616db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:28:17.322279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891f260-8eb1-433c-8e17-739279f616db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:17.323324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0dc7c-f89d-4e44-80df-26c3ca070a85 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:17.327468 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891f260-8eb1-433c-8e17-739279f616db admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:17.385485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 858/1739] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:28:17.400396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0dc7c-f89d-4e44-80df-26c3ca070a85 service nova] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:17.410735 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 882/1740] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:28:17.462279 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0dc7c-f89d-4e44-80df-26c3ca070a85 service nova] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:17.478509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf134211-4bef-41df-b432-7119da4a3724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:17.480628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf134211-4bef-41df-b432-7119da4a3724 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:17.481073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf134211-4bef-41df-b432-7119da4a3724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:17.482360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf134211-4bef-41df-b432-7119da4a3724 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:17.488480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf134211-4bef-41df-b432-7119da4a3724 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:17.490901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf134211-4bef-41df-b432-7119da4a3724 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:17.555287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 859/1741] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:17 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:17.562037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891f260-8eb1-433c-8e17-739279f616db admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:17.582849 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891f260-8eb1-433c-8e17-739279f616db admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:17.789173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:17.805299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfaf8a4-4d1d-4f38-a52e-dd624d039427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:17.806078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaf8a4-4d1d-4f38-a52e-dd624d039427 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:17.806455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaf8a4-4d1d-4f38-a52e-dd624d039427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:17.806968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaf8a4-4d1d-4f38-a52e-dd624d039427 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:17.807746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfaf8a4-4d1d-4f38-a52e-dd624d039427 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:17.810677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfaf8a4-4d1d-4f38-a52e-dd624d039427 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:17.850702 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 883/1742] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:17 2022] POST /identity/v3/users => generated 294 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:17.854515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf134211-4bef-41df-b432-7119da4a3724 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:17.870971 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf134211-4bef-41df-b432-7119da4a3724 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:17.888914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ef802-4c18-47ed-8f83-3f02b70bb8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:17.889552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ef802-4c18-47ed-8f83-3f02b70bb8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:17.889835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ef802-4c18-47ed-8f83-3f02b70bb8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:17.890112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ef802-4c18-47ed-8f83-3f02b70bb8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:17.890765 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ef802-4c18-47ed-8f83-3f02b70bb8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:17.892534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ef802-4c18-47ed-8f83-3f02b70bb8d1 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:17.921805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 860/1743] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:17.925980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfaf8a4-4d1d-4f38-a52e-dd624d039427 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:17.943888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfaf8a4-4d1d-4f38-a52e-dd624d039427 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:17.956175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70b687-b81f-4e1b-82e5-b44e2af95588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:17.956804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70b687-b81f-4e1b-82e5-b44e2af95588 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:17.957065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70b687-b81f-4e1b-82e5-b44e2af95588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:17.957337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70b687-b81f-4e1b-82e5-b44e2af95588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:17.957983 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70b687-b81f-4e1b-82e5-b44e2af95588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:17.959594 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70b687-b81f-4e1b-82e5-b44e2af95588 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:17.987882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 884/1744] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:17.993415 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ef802-4c18-47ed-8f83-3f02b70bb8d1 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:18.016371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ef802-4c18-47ed-8f83-3f02b70bb8d1 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:18.029746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106c497-d1f2-47e8-8bb5-2226db2fe3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:18.030524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106c497-d1f2-47e8-8bb5-2226db2fe3f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:18.030980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106c497-d1f2-47e8-8bb5-2226db2fe3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:18.031261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106c497-d1f2-47e8-8bb5-2226db2fe3f1 admin admin] PATH_INFO: `/v3/projects/9f689c5d29494f9b9e33284d6ccf9057/users/0f7c1ef83a154c15a3fdb90bb213f56b/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:18.069277 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106c497-d1f2-47e8-8bb5-2226db2fe3f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f689c5d29494f9b9e33284d6ccf9057, user_id=0f7c1ef83a154c15a3fdb90bb213f56b, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:18.073292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106c497-d1f2-47e8-8bb5-2226db2fe3f1 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:18.110503 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 861/1745] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:17 2022] PUT /identity/v3/projects/9f689c5d29494f9b9e33284d6ccf9057/users/0f7c1ef83a154c15a3fdb90bb213f56b/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:18.123550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a55ae-01c5-42ca-912c-dc09bef3cc88 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:18.124140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a55ae-01c5-42ca-912c-dc09bef3cc88 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:18.124433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a55ae-01c5-42ca-912c-dc09bef3cc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:18.200425 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bc7a55ae-01c5-42ca-912c-dc09bef3cc88 None None] MFA Rules not processed for user `0f7c1ef83a154c15a3fdb90bb213f56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:18.201689 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bc7a55ae-01c5-42ca-912c-dc09bef3cc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:18.359222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106c497-d1f2-47e8-8bb5-2226db2fe3f1 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:18.359910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 885/1746] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:18.518386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106c497-d1f2-47e8-8bb5-2226db2fe3f1 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:18.540875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e997720-3250-4415-b328-9e678c919521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:18.550463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e997720-3250-4415-b328-9e678c919521 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:18.550742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e997720-3250-4415-b328-9e678c919521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:18.551023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e997720-3250-4415-b328-9e678c919521 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:18.575864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e997720-3250-4415-b328-9e678c919521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:18.577949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e997720-3250-4415-b328-9e678c919521 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:18.659055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 862/1747] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:18 2022] POST /identity/v3/projects => generated 385 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:18.663289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a55ae-01c5-42ca-912c-dc09bef3cc88 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:18.676269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a55ae-01c5-42ca-912c-dc09bef3cc88 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:18.758509 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3fd02-c7c3-4795-986e-49bf1521cd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:18.759064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3fd02-c7c3-4795-986e-49bf1521cd64 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:18.759720 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3fd02-c7c3-4795-986e-49bf1521cd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:18.760005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3fd02-c7c3-4795-986e-49bf1521cd64 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:18.760655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3fd02-c7c3-4795-986e-49bf1521cd64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:18.762626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3fd02-c7c3-4795-986e-49bf1521cd64 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:18.862930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e997720-3250-4415-b328-9e678c919521 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:18.871017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 886/1748] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:18 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:18.877040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e997720-3250-4415-b328-9e678c919521 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:18.899878 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135ac1f-3120-4b81-811e-8bf675215770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:18.899878 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135ac1f-3120-4b81-811e-8bf675215770 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:18.899878 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135ac1f-3120-4b81-811e-8bf675215770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:18.899878 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135ac1f-3120-4b81-811e-8bf675215770 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:18.900287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135ac1f-3120-4b81-811e-8bf675215770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:18.900287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135ac1f-3120-4b81-811e-8bf675215770 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:18.908191 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3fd02-c7c3-4795-986e-49bf1521cd64 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:18.936732 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 863/1749] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:28:18.943292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3fd02-c7c3-4795-986e-49bf1521cd64 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:18.953672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15caeedf-f10c-4536-a330-32da04a14d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:18.954390 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caeedf-f10c-4536-a330-32da04a14d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:18.955164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caeedf-f10c-4536-a330-32da04a14d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:18.955444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caeedf-f10c-4536-a330-32da04a14d8b admin admin] PATH_INFO: `/v3/projects/607e03e585c84f2893a51549b5ce22a0/users/117adc9f965f43039b0e4b31e8809294/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:19.006185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15caeedf-f10c-4536-a330-32da04a14d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=607e03e585c84f2893a51549b5ce22a0, user_id=117adc9f965f43039b0e4b31e8809294, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:19.014290 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15caeedf-f10c-4536-a330-32da04a14d8b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:19.042295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135ac1f-3120-4b81-811e-8bf675215770 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:19.044393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-f135ac1f-3120-4b81-811e-8bf675215770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:19.072998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 887/1750] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:18 2022] PUT /identity/v3/projects/607e03e585c84f2893a51549b5ce22a0/users/117adc9f965f43039b0e4b31e8809294/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:19.074599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15caeedf-f10c-4536-a330-32da04a14d8b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:19.082416 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:19.201521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:19.267905 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135ac1f-3120-4b81-811e-8bf675215770 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:19.324865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15caeedf-f10c-4536-a330-32da04a14d8b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:19.342908 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb744452-52aa-4586-86d7-72ecb2b0b67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:19.343549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb744452-52aa-4586-86d7-72ecb2b0b67d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:19.343862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb744452-52aa-4586-86d7-72ecb2b0b67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:19.344140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb744452-52aa-4586-86d7-72ecb2b0b67d admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:19.344795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb744452-52aa-4586-86d7-72ecb2b0b67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:19.346957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb744452-52aa-4586-86d7-72ecb2b0b67d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:19.378159 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 888/1751] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:19.379686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6768d0d-0eae-438e-ad0e-5919ad5a53ef None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:19.379972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6768d0d-0eae-438e-ad0e-5919ad5a53ef None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:19.395608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6768d0d-0eae-438e-ad0e-5919ad5a53ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:19.465700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e6768d0d-0eae-438e-ad0e-5919ad5a53ef None None] MFA Rules not processed for user `98da6780db2d43d5be8e971dd2d86119`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:19.466844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e6768d0d-0eae-438e-ad0e-5919ad5a53ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:19.481750 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a01eb-1841-47ab-b8a7-5dc5bcfcb406 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b3b0a0317b14e29a32eb6ba6b89c2aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:19.482610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a01eb-1841-47ab-b8a7-5dc5bcfcb406 service placement] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:19.482994 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a01eb-1841-47ab-b8a7-5dc5bcfcb406 service placement] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:19.483342 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a01eb-1841-47ab-b8a7-5dc5bcfcb406 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:19.488735 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b85a01eb-1841-47ab-b8a7-5dc5bcfcb406 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:19.518055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a01eb-1841-47ab-b8a7-5dc5bcfcb406 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:19.519725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a01eb-1841-47ab-b8a7-5dc5bcfcb406 service placement] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:19.549146 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 864/1752] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:28:19.550080 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a01eb-1841-47ab-b8a7-5dc5bcfcb406 service placement] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:19.598016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a01eb-1841-47ab-b8a7-5dc5bcfcb406 service placement] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:19.614286 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485577f2-acba-47a0-b500-4e049c201f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:19.616528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485577f2-acba-47a0-b500-4e049c201f17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:19.617022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485577f2-acba-47a0-b500-4e049c201f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:19.617489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485577f2-acba-47a0-b500-4e049c201f17 admin admin] PATH_INFO: `/v3/projects/607e03e585c84f2893a51549b5ce22a0/users/117adc9f965f43039b0e4b31e8809294/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:19.621660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485577f2-acba-47a0-b500-4e049c201f17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=607e03e585c84f2893a51549b5ce22a0, user_id=117adc9f965f43039b0e4b31e8809294, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:19.621811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 889/1753] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:19.632997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e8a77-c5c0-45f9-8a87-15a85d64365e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:19.633288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e8a77-c5c0-45f9-8a87-15a85d64365e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:19.633564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e8a77-c5c0-45f9-8a87-15a85d64365e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:19.634678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485577f2-acba-47a0-b500-4e049c201f17 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:19.659246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 865/1754] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:19 2022] PUT /identity/v3/projects/607e03e585c84f2893a51549b5ce22a0/users/117adc9f965f43039b0e4b31e8809294/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:19.682546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c6a4f-f143-4c50-b15e-277e4ef2dfd3 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:19.683472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c6a4f-f143-4c50-b15e-277e4ef2dfd3 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:19.684029 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c6a4f-f143-4c50-b15e-277e4ef2dfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:19.697975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-398e8a77-c5c0-45f9-8a87-15a85d64365e None None] MFA Rules not processed for user `98da6780db2d43d5be8e971dd2d86119`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:19.702011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-398e8a77-c5c0-45f9-8a87-15a85d64365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:19.806606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-bc0c6a4f-f143-4c50-b15e-277e4ef2dfd3 None None] MFA Rules not processed for user `117adc9f965f43039b0e4b31e8809294`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:19.807928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bc0c6a4f-f143-4c50-b15e-277e4ef2dfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:19.912990 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 890/1755] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:28:19.915530 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e8a77-c5c0-45f9-8a87-15a85d64365e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:19.925205 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-398e8a77-c5c0-45f9-8a87-15a85d64365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:20.014317 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 866/1756] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:20.021863 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64484bf6-11a8-44d6-826d-967360243a47 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:20.022158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64484bf6-11a8-44d6-826d-967360243a47 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:20.022433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64484bf6-11a8-44d6-826d-967360243a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:20.070894 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-64484bf6-11a8-44d6-826d-967360243a47 None None] MFA Rules not processed for user `7f56dde21c754cfe8253ec5c1b7b90d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:20.072169 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-64484bf6-11a8-44d6-826d-967360243a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:20.088133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:20.095055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:20.099356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e8a77-c5c0-45f9-8a87-15a85d64365e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:20.110250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:20.111781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:20.112206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:20.112620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] PATH_INFO: `/v3/projects/b7337005f99b48a5896a97ce1bd20fa5` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:20.114046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7337005f99b48a5896a97ce1bd20fa5)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:20.117214 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:20.221977 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 867/1757] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:20.223788 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69363b3a-7b59-4cbf-a402-e7573fa114be None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:20.224125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69363b3a-7b59-4cbf-a402-e7573fa114be None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:20.224410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69363b3a-7b59-4cbf-a402-e7573fa114be None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:20.233687 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 868/1758] 10.222.0.27 () {54 vars in 912 bytes} [Mon Sep 12 23:28:20 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:28:20.234041 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7337005f99b48a5896a97ce1bd20fa5', 'request_id': 'req-fd91276e-bac4-4b2d-b550-b95a0a23453a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:20.234651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69363b3a-7b59-4cbf-a402-e7573fa114be None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:20.236754 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-69363b3a-7b59-4cbf-a402-e7573fa114be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:20.241955 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7337005f99b48a5896a97ce1bd20fa5', 'request_id': 'req-fd91276e-bac4-4b2d-b550-b95a0a23453a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:20.252075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7337005f99b48a5896a97ce1bd20fa5', 'request_id': 'req-fd91276e-bac4-4b2d-b550-b95a0a23453a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:20.270793 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] The token cache is being invalidate because project b7337005f99b48a5896a97ce1bd20fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:20.271130 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:20.279446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:20.287883 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 891/1759] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:19 2022] DELETE /identity/v3/projects/b7337005f99b48a5896a97ce1bd20fa5 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:20.290896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:20.298196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:20.442065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69363b3a-7b59-4cbf-a402-e7573fa114be None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:20.461316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad321e-602d-4323-9741-c3091af35e2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:20.461954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad321e-602d-4323-9741-c3091af35e2c service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:20.462221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad321e-602d-4323-9741-c3091af35e2c service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:20.462497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad321e-602d-4323-9741-c3091af35e2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:20.465175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6fad321e-602d-4323-9741-c3091af35e2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:20.483969 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91276e-bac4-4b2d-b550-b95a0a23453a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:20.493643 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f53e14-341b-4519-9dcf-efb2c622a3c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:20.494734 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f53e14-341b-4519-9dcf-efb2c622a3c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:20.495268 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f53e14-341b-4519-9dcf-efb2c622a3c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:20.495813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f53e14-341b-4519-9dcf-efb2c622a3c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:20.500955 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-60f53e14-341b-4519-9dcf-efb2c622a3c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:20.598329 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad321e-602d-4323-9741-c3091af35e2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:20.601614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad321e-602d-4323-9741-c3091af35e2c service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:20.631507 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 869/1760] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:20.637719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c154987-1756-4031-b9a0-1279423667c7 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:20.639237 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c154987-1756-4031-b9a0-1279423667c7 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:20.639551 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c154987-1756-4031-b9a0-1279423667c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:20.703413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f53e14-341b-4519-9dcf-efb2c622a3c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:20.705441 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f53e14-341b-4519-9dcf-efb2c622a3c0 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:20.722323 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-4c154987-1756-4031-b9a0-1279423667c7 None None] MFA Rules not processed for user `bc6e63e9fc3447d5a894a777a98fe773`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:20.725178 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4c154987-1756-4031-b9a0-1279423667c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:20.740582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 892/1761] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:20.743321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3654cb-9711-41d0-882d-15155cbd95c7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:20.743636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3654cb-9711-41d0-882d-15155cbd95c7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:20.743921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3654cb-9711-41d0-882d-15155cbd95c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:20.819663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2b3654cb-9711-41d0-882d-15155cbd95c7 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:20.821051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2b3654cb-9711-41d0-882d-15155cbd95c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:20.867278 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:20.957521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 893/1762] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:20.961736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 870/1763] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:28:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:20.966658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df310a-6af9-4c70-800b-4680a08815d1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:20.966945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df310a-6af9-4c70-800b-4680a08815d1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:20.967226 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df310a-6af9-4c70-800b-4680a08815d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:20.975387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c154987-1756-4031-b9a0-1279423667c7 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:20.976911 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4c154987-1756-4031-b9a0-1279423667c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:21.004392 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c154987-1756-4031-b9a0-1279423667c7 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:21.015820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b83636-2c4d-4b47-8fcb-a3e114395e71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:21.016461 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b83636-2c4d-4b47-8fcb-a3e114395e71 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:21.016745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b83636-2c4d-4b47-8fcb-a3e114395e71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:21.017589 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b83636-2c4d-4b47-8fcb-a3e114395e71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:21.023320 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:21.025869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b83636-2c4d-4b47-8fcb-a3e114395e71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:21.028054 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b83636-2c4d-4b47-8fcb-a3e114395e71 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:21.032139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-17df310a-6af9-4c70-800b-4680a08815d1 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:21.034164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-17df310a-6af9-4c70-800b-4680a08815d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:21.044901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 871/1764] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:21.054984 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 894/1765] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:21.056580 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b83636-2c4d-4b47-8fcb-a3e114395e71 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:21.081177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b83636-2c4d-4b47-8fcb-a3e114395e71 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:21.093056 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791471a-2ac6-4160-b8ef-5d1b9c8556bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:21.095050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791471a-2ac6-4160-b8ef-5d1b9c8556bf admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:21.095050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791471a-2ac6-4160-b8ef-5d1b9c8556bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:21.095050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791471a-2ac6-4160-b8ef-5d1b9c8556bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:21.095736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791471a-2ac6-4160-b8ef-5d1b9c8556bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:28:21.095915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791471a-2ac6-4160-b8ef-5d1b9c8556bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:21.097889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791471a-2ac6-4160-b8ef-5d1b9c8556bf admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:21.139018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 872/1766] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:28:21.140304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df310a-6af9-4c70-800b-4680a08815d1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:21.159991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df310a-6af9-4c70-800b-4680a08815d1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:21.189998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d4463-2806-474c-abeb-0f278cabcf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:21.190625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d4463-2806-474c-abeb-0f278cabcf3d admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:21.190904 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d4463-2806-474c-abeb-0f278cabcf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:21.191377 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d4463-2806-474c-abeb-0f278cabcf3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:21.194654 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d4463-2806-474c-abeb-0f278cabcf3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:21.200660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d4463-2806-474c-abeb-0f278cabcf3d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:21.265300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 895/1767] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:21 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:21.269892 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791471a-2ac6-4160-b8ef-5d1b9c8556bf admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:21.294475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791471a-2ac6-4160-b8ef-5d1b9c8556bf admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:21.385241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f03c51-0741-42ae-92ec-157297df3166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:21.385934 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f03c51-0741-42ae-92ec-157297df3166 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:21.386225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f03c51-0741-42ae-92ec-157297df3166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:21.386501 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f03c51-0741-42ae-92ec-157297df3166 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:21.387355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f03c51-0741-42ae-92ec-157297df3166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:21.389496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f03c51-0741-42ae-92ec-157297df3166 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:21.427227 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 873/1768] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:21 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:21.427457 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d4463-2806-474c-abeb-0f278cabcf3d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:21.444702 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d4463-2806-474c-abeb-0f278cabcf3d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:21.452978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e054a-8e72-4ae2-b4f5-913d24c0ca28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:21.453601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e054a-8e72-4ae2-b4f5-913d24c0ca28 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:21.453882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e054a-8e72-4ae2-b4f5-913d24c0ca28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:21.454157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e054a-8e72-4ae2-b4f5-913d24c0ca28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:21.455071 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e054a-8e72-4ae2-b4f5-913d24c0ca28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:21.457243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e054a-8e72-4ae2-b4f5-913d24c0ca28 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:21.480672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 896/1769] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:28:21.484478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f03c51-0741-42ae-92ec-157297df3166 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:21.515010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f03c51-0741-42ae-92ec-157297df3166 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:21.528633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b3846-ac4f-4dbe-80eb-a63558f433a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:21.529403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b3846-ac4f-4dbe-80eb-a63558f433a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:21.530207 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b3846-ac4f-4dbe-80eb-a63558f433a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:21.531824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b3846-ac4f-4dbe-80eb-a63558f433a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:21.534107 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b3846-ac4f-4dbe-80eb-a63558f433a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:21.540164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b3846-ac4f-4dbe-80eb-a63558f433a9 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:21.577612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 874/1770] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:21.583219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e054a-8e72-4ae2-b4f5-913d24c0ca28 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:21.598656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e054a-8e72-4ae2-b4f5-913d24c0ca28 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:21.610516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90f3ec-ff8f-49e8-a607-7241e598fcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:21.611586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90f3ec-ff8f-49e8-a607-7241e598fcd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:21.612696 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90f3ec-ff8f-49e8-a607-7241e598fcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:21.615533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90f3ec-ff8f-49e8-a607-7241e598fcd9 admin admin] PATH_INFO: `/v3/projects/9f79837133d94ceea715cf1c44716924/users/7be0f2d4f7344ccba50c3e86f478e568/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:21.677851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90f3ec-ff8f-49e8-a607-7241e598fcd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f79837133d94ceea715cf1c44716924, user_id=7be0f2d4f7344ccba50c3e86f478e568, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:21.692723 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90f3ec-ff8f-49e8-a607-7241e598fcd9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:21.725270 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 897/1771] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:21 2022] PUT /identity/v3/projects/9f79837133d94ceea715cf1c44716924/users/7be0f2d4f7344ccba50c3e86f478e568/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:21.747190 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ffd1e-8e08-49f0-82eb-194ce47741ed None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:21.747614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ffd1e-8e08-49f0-82eb-194ce47741ed None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:21.748181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ffd1e-8e08-49f0-82eb-194ce47741ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:21.794293 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-4c9ffd1e-8e08-49f0-82eb-194ce47741ed None None] MFA Rules not processed for user `7be0f2d4f7344ccba50c3e86f478e568`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:21.799653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4c9ffd1e-8e08-49f0-82eb-194ce47741ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:22.001036 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90f3ec-ff8f-49e8-a607-7241e598fcd9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:22.004612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bc90f3ec-ff8f-49e8-a607-7241e598fcd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:22.023424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 875/1772] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:22.024744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ffd1e-8e08-49f0-82eb-194ce47741ed None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:22.227066 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90f3ec-ff8f-49e8-a607-7241e598fcd9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:22.240844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f92c10-0cc6-449e-b81b-466587da4811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:22.243707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f92c10-0cc6-449e-b81b-466587da4811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:22.244016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f92c10-0cc6-449e-b81b-466587da4811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:22.244442 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f92c10-0cc6-449e-b81b-466587da4811 admin admin] PATH_INFO: `/v3/users/0f7c1ef83a154c15a3fdb90bb213f56b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:22.246313 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:22.247593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f92c10-0cc6-449e-b81b-466587da4811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f7c1ef83a154c15a3fdb90bb213f56b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:22.253612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f92c10-0cc6-449e-b81b-466587da4811 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:22.298770 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ffd1e-8e08-49f0-82eb-194ce47741ed None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:22.319733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850cbcc-cde0-4a51-9ede-2dab09094a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:22.320610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850cbcc-cde0-4a51-9ede-2dab09094a74 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:22.321370 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850cbcc-cde0-4a51-9ede-2dab09094a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:22.321370 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850cbcc-cde0-4a51-9ede-2dab09094a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:22.322050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850cbcc-cde0-4a51-9ede-2dab09094a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:22.328869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850cbcc-cde0-4a51-9ede-2dab09094a74 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:22.394456 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-31f92c10-0cc6-449e-b81b-466587da4811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f7c1ef83a154c15a3fdb90bb213f56b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:22.411605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 876/1773] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:22 2022] POST /identity/v3/projects => generated 383 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:22.415261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850cbcc-cde0-4a51-9ede-2dab09094a74 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:22.419521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-31f92c10-0cc6-449e-b81b-466587da4811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f7c1ef83a154c15a3fdb90bb213f56b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:22.478460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-31f92c10-0cc6-449e-b81b-466587da4811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f7c1ef83a154c15a3fdb90bb213f56b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:22.484820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:22.499672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-31f92c10-0cc6-449e-b81b-466587da4811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f7c1ef83a154c15a3fdb90bb213f56b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:22.501755 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-31f92c10-0cc6-449e-b81b-466587da4811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f7c1ef83a154c15a3fdb90bb213f56b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:22.521357 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 898/1774] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:21 2022] DELETE /identity/v3/users/0f7c1ef83a154c15a3fdb90bb213f56b => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:22.524078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b70e0-10af-415b-829e-a69d24ef0c49 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:22.524510 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b70e0-10af-415b-829e-a69d24ef0c49 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:22.525097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b70e0-10af-415b-829e-a69d24ef0c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:22.570359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-cc1b70e0-10af-415b-829e-a69d24ef0c49 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:22.572229 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cc1b70e0-10af-415b-829e-a69d24ef0c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:22.597069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850cbcc-cde0-4a51-9ede-2dab09094a74 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:22.691846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bc7af-6f2d-4e2e-ba64-00c9acd6cd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:22.692513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bc7af-6f2d-4e2e-ba64-00c9acd6cd5a admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:22.692797 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bc7af-6f2d-4e2e-ba64-00c9acd6cd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:22.693613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bc7af-6f2d-4e2e-ba64-00c9acd6cd5a admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:22.693933 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bc7af-6f2d-4e2e-ba64-00c9acd6cd5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:22.696337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bc7af-6f2d-4e2e-ba64-00c9acd6cd5a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:22.739649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 899/1775] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:22.747851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 877/1776] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:22 2022] POST /identity/v3/users => generated 292 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:22.748301 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b70e0-10af-415b-829e-a69d24ef0c49 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:22.750175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cc1b70e0-10af-415b-829e-a69d24ef0c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:22.778125 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b70e0-10af-415b-829e-a69d24ef0c49 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:22.792396 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adead69c-b7d7-415b-a710-6443098555cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:22.792962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adead69c-b7d7-415b-a710-6443098555cc admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:22.793236 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adead69c-b7d7-415b-a710-6443098555cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:22.793612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adead69c-b7d7-415b-a710-6443098555cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:22.795635 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bc7af-6f2d-4e2e-ba64-00c9acd6cd5a admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:22.797020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adead69c-b7d7-415b-a710-6443098555cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:22.797827 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-683bc7af-6f2d-4e2e-ba64-00c9acd6cd5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:22.801637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adead69c-b7d7-415b-a710-6443098555cc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:22.827894 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 900/1777] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:22.831458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adead69c-b7d7-415b-a710-6443098555cc admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:22.851788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adead69c-b7d7-415b-a710-6443098555cc admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:22.870824 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3ab19-e823-458d-bd39-6b11a95f0165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:22.875659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3ab19-e823-458d-bd39-6b11a95f0165 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:22.876102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3ab19-e823-458d-bd39-6b11a95f0165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:22.877012 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3ab19-e823-458d-bd39-6b11a95f0165 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:22.879423 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3ab19-e823-458d-bd39-6b11a95f0165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:22.888147 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3ab19-e823-458d-bd39-6b11a95f0165 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:22.940587 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 901/1778] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:22.942049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9ae33-17db-45e5-b2ce-74fe0b924d68 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:22.942442 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9ae33-17db-45e5-b2ce-74fe0b924d68 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:22.942807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9ae33-17db-45e5-b2ce-74fe0b924d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:22.997181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bc7af-6f2d-4e2e-ba64-00c9acd6cd5a admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:23.005927 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bad9ae33-17db-45e5-b2ce-74fe0b924d68 None None] MFA Rules not processed for user `0248519bf5b54ce3ba3b40f2c9a4c628`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:23.007863 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bad9ae33-17db-45e5-b2ce-74fe0b924d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:23.013848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914029de-fa4f-48f5-b79c-bdc8d8f602be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:23.017638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914029de-fa4f-48f5-b79c-bdc8d8f602be service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:23.018224 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914029de-fa4f-48f5-b79c-bdc8d8f602be service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:23.018648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914029de-fa4f-48f5-b79c-bdc8d8f602be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:23.044552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914029de-fa4f-48f5-b79c-bdc8d8f602be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:23.047885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914029de-fa4f-48f5-b79c-bdc8d8f602be service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:23.089454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 878/1779] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:28:23.090330 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914029de-fa4f-48f5-b79c-bdc8d8f602be service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:23.119766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914029de-fa4f-48f5-b79c-bdc8d8f602be service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:23.134152 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e920d38-a2ce-495d-b2a0-65e6547651d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:23.136335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e920d38-a2ce-495d-b2a0-65e6547651d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:23.136639 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e920d38-a2ce-495d-b2a0-65e6547651d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:23.136919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e920d38-a2ce-495d-b2a0-65e6547651d2 admin admin] PATH_INFO: `/v3/projects/c2e2315683c440c4b2d4e6a981c2b054/users/b763d6c5c2c545f69a7106a9162c6a23/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:23.196214 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:23.204824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e920d38-a2ce-495d-b2a0-65e6547651d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e2315683c440c4b2d4e6a981c2b054, user_id=b763d6c5c2c545f69a7106a9162c6a23, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:23.209648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e920d38-a2ce-495d-b2a0-65e6547651d2 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:23.225253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 902/1780] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:23.233563 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 879/1781] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:23 2022] PUT /identity/v3/projects/c2e2315683c440c4b2d4e6a981c2b054/users/b763d6c5c2c545f69a7106a9162c6a23/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:23.244610 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93742-da0f-4dd9-a8d0-2056f19a12ed None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:23.244896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93742-da0f-4dd9-a8d0-2056f19a12ed None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:23.245171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93742-da0f-4dd9-a8d0-2056f19a12ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:23.343558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dce93742-da0f-4dd9-a8d0-2056f19a12ed None None] MFA Rules not processed for user `b763d6c5c2c545f69a7106a9162c6a23`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:23.346451 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dce93742-da0f-4dd9-a8d0-2056f19a12ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:23.545135 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 903/1782] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:23.549988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e920d38-a2ce-495d-b2a0-65e6547651d2 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:23.679628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e920d38-a2ce-495d-b2a0-65e6547651d2 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:23.686100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954c8d5-c49f-428e-a677-09a164443aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:23.686795 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954c8d5-c49f-428e-a677-09a164443aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:23.687102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954c8d5-c49f-428e-a677-09a164443aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:23.687373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954c8d5-c49f-428e-a677-09a164443aa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:23.688186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954c8d5-c49f-428e-a677-09a164443aa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:23.690649 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954c8d5-c49f-428e-a677-09a164443aa5 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:23.740186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 880/1783] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:23 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:23.744102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce93742-da0f-4dd9-a8d0-2056f19a12ed None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:23.761149 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:23.771458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce93742-da0f-4dd9-a8d0-2056f19a12ed None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:23.936203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bba88-ddd2-413e-8e38-0489c366107f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:23.936873 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bba88-ddd2-413e-8e38-0489c366107f admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:23.937157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bba88-ddd2-413e-8e38-0489c366107f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:23.937828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bba88-ddd2-413e-8e38-0489c366107f admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:23.944779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969bba88-ddd2-413e-8e38-0489c366107f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:23.946925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969bba88-ddd2-413e-8e38-0489c366107f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:23.997525 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 904/1784] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:23 2022] POST /identity/v3/users => generated 299 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:23.999119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954c8d5-c49f-428e-a677-09a164443aa5 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:24.022055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954c8d5-c49f-428e-a677-09a164443aa5 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:24.033368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08dddc-982f-441b-a21b-b457b9cd9009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:24.035145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08dddc-982f-441b-a21b-b457b9cd9009 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:24.035436 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08dddc-982f-441b-a21b-b457b9cd9009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:24.037059 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08dddc-982f-441b-a21b-b457b9cd9009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:24.037791 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08dddc-982f-441b-a21b-b457b9cd9009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:24.041669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08dddc-982f-441b-a21b-b457b9cd9009 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:24.071801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 881/1785] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:24.072660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bba88-ddd2-413e-8e38-0489c366107f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:24.074545 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08dddc-982f-441b-a21b-b457b9cd9009 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:24.074667 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-969bba88-ddd2-413e-8e38-0489c366107f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:24.096736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08dddc-982f-441b-a21b-b457b9cd9009 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:24.101914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bba88-ddd2-413e-8e38-0489c366107f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:24.102861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78d294-a76c-4c46-8757-9b8e0eb6b855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:24.103931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78d294-a76c-4c46-8757-9b8e0eb6b855 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:24.104360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78d294-a76c-4c46-8757-9b8e0eb6b855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:24.104920 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78d294-a76c-4c46-8757-9b8e0eb6b855 admin admin] PATH_INFO: `/v3/projects/daa2a66bfcbd41e6aa49510abf1f4b9e/users/62137425104c4acf8251bc34aa870c79/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:24.133906 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78d294-a76c-4c46-8757-9b8e0eb6b855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa2a66bfcbd41e6aa49510abf1f4b9e, user_id=62137425104c4acf8251bc34aa870c79, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:24.134126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:24.134770 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:24.135056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:24.135340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] PATH_INFO: `/v3/projects/9f689c5d29494f9b9e33284d6ccf9057` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:24.138895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78d294-a76c-4c46-8757-9b8e0eb6b855 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:24.141067 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f689c5d29494f9b9e33284d6ccf9057)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:24.143542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:24.209240 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 882/1786] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:24 2022] PUT /identity/v3/projects/daa2a66bfcbd41e6aa49510abf1f4b9e/users/62137425104c4acf8251bc34aa870c79/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:24.215951 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78d294-a76c-4c46-8757-9b8e0eb6b855 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:24.321342 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f689c5d29494f9b9e33284d6ccf9057', 'request_id': 'req-355215ad-1c69-402f-afa7-5e8e7c250101'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:24.342739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f689c5d29494f9b9e33284d6ccf9057', 'request_id': 'req-355215ad-1c69-402f-afa7-5e8e7c250101'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:24.354044 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78d294-a76c-4c46-8757-9b8e0eb6b855 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:24.355240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f689c5d29494f9b9e33284d6ccf9057', 'request_id': 'req-355215ad-1c69-402f-afa7-5e8e7c250101'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:24.377295 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] The token cache is being invalidate because project 9f689c5d29494f9b9e33284d6ccf9057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:24.377721 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:24.378585 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:24.383191 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 905/1787] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:24 2022] DELETE /identity/v3/projects/9f689c5d29494f9b9e33284d6ccf9057 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:24.384238 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:24.385740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:24.434391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:24.456233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239887ff-fa4d-4cc9-80cf-806d5a0d3d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:24.456869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239887ff-fa4d-4cc9-80cf-806d5a0d3d94 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:24.457877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239887ff-fa4d-4cc9-80cf-806d5a0d3d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:24.458115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239887ff-fa4d-4cc9-80cf-806d5a0d3d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:24.458781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239887ff-fa4d-4cc9-80cf-806d5a0d3d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:24.460583 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239887ff-fa4d-4cc9-80cf-806d5a0d3d94 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:24.473939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 883/1788] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:24.478883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239887ff-fa4d-4cc9-80cf-806d5a0d3d94 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:24.481631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-239887ff-fa4d-4cc9-80cf-806d5a0d3d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:24.496404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239887ff-fa4d-4cc9-80cf-806d5a0d3d94 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:24.502323 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e6b90-0a55-459c-8da9-171a2f0821c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:24.503073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e6b90-0a55-459c-8da9-171a2f0821c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:24.503432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e6b90-0a55-459c-8da9-171a2f0821c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:24.503810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e6b90-0a55-459c-8da9-171a2f0821c1 admin admin] PATH_INFO: `/v3/projects/daa2a66bfcbd41e6aa49510abf1f4b9e/users/62137425104c4acf8251bc34aa870c79/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:24.505889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e6b90-0a55-459c-8da9-171a2f0821c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa2a66bfcbd41e6aa49510abf1f4b9e, user_id=62137425104c4acf8251bc34aa870c79, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:24.508974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e6b90-0a55-459c-8da9-171a2f0821c1 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:24.535621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 884/1789] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:24 2022] PUT /identity/v3/projects/daa2a66bfcbd41e6aa49510abf1f4b9e/users/62137425104c4acf8251bc34aa870c79/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:28:24.544263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355215ad-1c69-402f-afa7-5e8e7c250101 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:24.550861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed152aad-a0a2-48d0-baa4-9f417eaf23be None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:24.550861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed152aad-a0a2-48d0-baa4-9f417eaf23be None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:24.550861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed152aad-a0a2-48d0-baa4-9f417eaf23be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:24.584047 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:24.663159 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-ed152aad-a0a2-48d0-baa4-9f417eaf23be None None] MFA Rules not processed for user `62137425104c4acf8251bc34aa870c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:24.664434 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ed152aad-a0a2-48d0-baa4-9f417eaf23be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:24.674282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce194a5d-45be-4dda-bda4-ad8dc3a2c7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:24.676645 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce194a5d-45be-4dda-bda4-ad8dc3a2c7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:24.676936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce194a5d-45be-4dda-bda4-ad8dc3a2c7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:24.679200 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce194a5d-45be-4dda-bda4-ad8dc3a2c7c1 admin admin] PATH_INFO: `/v3/users/117adc9f965f43039b0e4b31e8809294` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:24.680949 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce194a5d-45be-4dda-bda4-ad8dc3a2c7c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=117adc9f965f43039b0e4b31e8809294)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:24.684543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce194a5d-45be-4dda-bda4-ad8dc3a2c7c1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:24.779239 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ce194a5d-45be-4dda-bda4-ad8dc3a2c7c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '117adc9f965f43039b0e4b31e8809294'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:24.791543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ce194a5d-45be-4dda-bda4-ad8dc3a2c7c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '117adc9f965f43039b0e4b31e8809294'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:24.807395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ce194a5d-45be-4dda-bda4-ad8dc3a2c7c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '117adc9f965f43039b0e4b31e8809294'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:24.808671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 885/1790] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:24.821009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed152aad-a0a2-48d0-baa4-9f417eaf23be None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:24.822836 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ed152aad-a0a2-48d0-baa4-9f417eaf23be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:24.826002 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ce194a5d-45be-4dda-bda4-ad8dc3a2c7c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '117adc9f965f43039b0e4b31e8809294'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:24.828073 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ce194a5d-45be-4dda-bda4-ad8dc3a2c7c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '117adc9f965f43039b0e4b31e8809294'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:24.846593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 906/1791] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:24 2022] DELETE /identity/v3/users/117adc9f965f43039b0e4b31e8809294 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:24.906057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed152aad-a0a2-48d0-baa4-9f417eaf23be None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:25.006365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae4d61-57e5-4655-a6b8-d69fc2d76594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:25.007204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae4d61-57e5-4655-a6b8-d69fc2d76594 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:25.007655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae4d61-57e5-4655-a6b8-d69fc2d76594 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:25.008111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae4d61-57e5-4655-a6b8-d69fc2d76594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:25.010347 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-fcae4d61-57e5-4655-a6b8-d69fc2d76594 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:25.049340 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:25.127373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae4d61-57e5-4655-a6b8-d69fc2d76594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:25.129686 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae4d61-57e5-4655-a6b8-d69fc2d76594 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:25.145821 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 886/1792] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:25.404408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce194a5d-45be-4dda-bda4-ad8dc3a2c7c1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:25.406404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ce194a5d-45be-4dda-bda4-ad8dc3a2c7c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:25.547268 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce194a5d-45be-4dda-bda4-ad8dc3a2c7c1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:25.553626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b6db7-6f34-4662-a797-06f785af4eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:25.554361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b6db7-6f34-4662-a797-06f785af4eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:25.554735 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b6db7-6f34-4662-a797-06f785af4eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:25.555092 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b6db7-6f34-4662-a797-06f785af4eee admin admin] PATH_INFO: `/v3/projects/607e03e585c84f2893a51549b5ce22a0` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:25.556642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b6db7-6f34-4662-a797-06f785af4eee admin admin] RBAC: Authorizing `identity:delete_project(project_id=607e03e585c84f2893a51549b5ce22a0)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:25.559441 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b6db7-6f34-4662-a797-06f785af4eee admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:25.617847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:25.621190 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8c5b6db7-6f34-4662-a797-06f785af4eee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '607e03e585c84f2893a51549b5ce22a0', 'request_id': 'req-8c5b6db7-6f34-4662-a797-06f785af4eee'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:25.636439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8c5b6db7-6f34-4662-a797-06f785af4eee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '607e03e585c84f2893a51549b5ce22a0', 'request_id': 'req-8c5b6db7-6f34-4662-a797-06f785af4eee'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:25.648188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8c5b6db7-6f34-4662-a797-06f785af4eee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '607e03e585c84f2893a51549b5ce22a0', 'request_id': 'req-8c5b6db7-6f34-4662-a797-06f785af4eee'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:25.669883 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8c5b6db7-6f34-4662-a797-06f785af4eee admin admin] The token cache is being invalidate because project 607e03e585c84f2893a51549b5ce22a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:25.670253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8c5b6db7-6f34-4662-a797-06f785af4eee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:25.671144 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8c5b6db7-6f34-4662-a797-06f785af4eee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:25.673299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 907/1793] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:25 2022] DELETE /identity/v3/projects/607e03e585c84f2893a51549b5ce22a0 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:25.677966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738291b2-d18b-4956-9bdd-258e9eb9140b None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:25.678383 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738291b2-d18b-4956-9bdd-258e9eb9140b None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:25.678530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738291b2-d18b-4956-9bdd-258e9eb9140b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:25.702612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-738291b2-d18b-4956-9bdd-258e9eb9140b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:25.703849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-738291b2-d18b-4956-9bdd-258e9eb9140b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:25.775169 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 887/1794] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:25.785496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdb2d6-d5e8-43de-b194-e6f2af08c927 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:25.785822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdb2d6-d5e8-43de-b194-e6f2af08c927 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:25.786131 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdb2d6-d5e8-43de-b194-e6f2af08c927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:25.818401 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2acdb2d6-d5e8-43de-b194-e6f2af08c927 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:25.819786 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2acdb2d6-d5e8-43de-b194-e6f2af08c927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:25.827809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 908/1795] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:25.832254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738291b2-d18b-4956-9bdd-258e9eb9140b None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:25.846289 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738291b2-d18b-4956-9bdd-258e9eb9140b None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:25.851606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09163776-5155-486c-8bf2-7b5d1c5fd3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:25.852165 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09163776-5155-486c-8bf2-7b5d1c5fd3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:25.852628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09163776-5155-486c-8bf2-7b5d1c5fd3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:25.852912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09163776-5155-486c-8bf2-7b5d1c5fd3d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:25.854050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09163776-5155-486c-8bf2-7b5d1c5fd3d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:28:25.854050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09163776-5155-486c-8bf2-7b5d1c5fd3d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:25.856067 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09163776-5155-486c-8bf2-7b5d1c5fd3d4 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:25.873640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 888/1796] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:28:25.878215 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdb2d6-d5e8-43de-b194-e6f2af08c927 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:25.889474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdb2d6-d5e8-43de-b194-e6f2af08c927 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:25.894464 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab4de8-d2f9-40f1-a9b4-85dfcb30db94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:25.895075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4de8-d2f9-40f1-a9b4-85dfcb30db94 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:25.896060 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4de8-d2f9-40f1-a9b4-85dfcb30db94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:25.896060 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4de8-d2f9-40f1-a9b4-85dfcb30db94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:25.896363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab4de8-d2f9-40f1-a9b4-85dfcb30db94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:25.898418 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab4de8-d2f9-40f1-a9b4-85dfcb30db94 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:25.948625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 909/1797] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:25 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:25.949890 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09163776-5155-486c-8bf2-7b5d1c5fd3d4 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:25.960029 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09163776-5155-486c-8bf2-7b5d1c5fd3d4 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:26.019085 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab4de8-d2f9-40f1-a9b4-85dfcb30db94 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:26.022082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7eab4de8-d2f9-40f1-a9b4-85dfcb30db94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:26.094452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518e522-a3e1-41d1-a6d7-4f4871508784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:26.096499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518e522-a3e1-41d1-a6d7-4f4871508784 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:26.096781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518e522-a3e1-41d1-a6d7-4f4871508784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:26.097052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518e522-a3e1-41d1-a6d7-4f4871508784 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:26.097686 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518e522-a3e1-41d1-a6d7-4f4871508784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:26.108185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518e522-a3e1-41d1-a6d7-4f4871508784 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:26.137324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 889/1798] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:25 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:26.141193 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518e522-a3e1-41d1-a6d7-4f4871508784 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:26.163479 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518e522-a3e1-41d1-a6d7-4f4871508784 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:26.181403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549599e1-aa02-4ef6-a23b-a0413e07cd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:26.181834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549599e1-aa02-4ef6-a23b-a0413e07cd55 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:26.182276 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549599e1-aa02-4ef6-a23b-a0413e07cd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:26.184483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549599e1-aa02-4ef6-a23b-a0413e07cd55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:26.185313 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549599e1-aa02-4ef6-a23b-a0413e07cd55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:26.187226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549599e1-aa02-4ef6-a23b-a0413e07cd55 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:26.202354 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab4de8-d2f9-40f1-a9b4-85dfcb30db94 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:26.206865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13096f20-95fb-4a47-8a25-9a53b3456250 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:26.207389 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13096f20-95fb-4a47-8a25-9a53b3456250 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:26.207822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13096f20-95fb-4a47-8a25-9a53b3456250 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:26.208104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13096f20-95fb-4a47-8a25-9a53b3456250 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:26.209909 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-13096f20-95fb-4a47-8a25-9a53b3456250 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:26.227864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 890/1799] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:28:26.229028 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549599e1-aa02-4ef6-a23b-a0413e07cd55 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:26.257074 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549599e1-aa02-4ef6-a23b-a0413e07cd55 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:26.264307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606413a7-ab02-424d-bf77-6a2ea7584317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:26.265172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606413a7-ab02-424d-bf77-6a2ea7584317 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:26.277271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606413a7-ab02-424d-bf77-6a2ea7584317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:26.277846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606413a7-ab02-424d-bf77-6a2ea7584317 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:26.288804 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606413a7-ab02-424d-bf77-6a2ea7584317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:26.290480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606413a7-ab02-424d-bf77-6a2ea7584317 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:26.322901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 891/1800] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:26.327516 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606413a7-ab02-424d-bf77-6a2ea7584317 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:26.349203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606413a7-ab02-424d-bf77-6a2ea7584317 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:26.363738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be308795-8eac-4541-b33a-b74322fbc276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:26.364575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be308795-8eac-4541-b33a-b74322fbc276 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:26.364994 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be308795-8eac-4541-b33a-b74322fbc276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:26.365276 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be308795-8eac-4541-b33a-b74322fbc276 admin admin] PATH_INFO: `/v3/projects/83ff5deecb06424b8abf31397c5af0a0/users/9f5a9cd2a87f480aa8a34b6bd90cbe48/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:26.397409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:26.419970 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be308795-8eac-4541-b33a-b74322fbc276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ff5deecb06424b8abf31397c5af0a0, user_id=9f5a9cd2a87f480aa8a34b6bd90cbe48, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:26.424980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13096f20-95fb-4a47-8a25-9a53b3456250 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:26.427116 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13096f20-95fb-4a47-8a25-9a53b3456250 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:26.429286 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be308795-8eac-4541-b33a-b74322fbc276 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:26.452157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 910/1801] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:28:26.474920 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 892/1802] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:26 2022] PUT /identity/v3/projects/83ff5deecb06424b8abf31397c5af0a0/users/9f5a9cd2a87f480aa8a34b6bd90cbe48/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:26.484508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b410e6d-ed80-4bf4-acd0-88bfc7f29991 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:26.487887 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b410e6d-ed80-4bf4-acd0-88bfc7f29991 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:26.488168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b410e6d-ed80-4bf4-acd0-88bfc7f29991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:26.550815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9b410e6d-ed80-4bf4-acd0-88bfc7f29991 None None] MFA Rules not processed for user `9f5a9cd2a87f480aa8a34b6bd90cbe48`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:26.552780 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9b410e6d-ed80-4bf4-acd0-88bfc7f29991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:26.654511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:26.694715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 911/1803] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:26.695816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c0a72-992e-41e0-af95-ba7454891140 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:26.696692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c0a72-992e-41e0-af95-ba7454891140 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:26.697230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c0a72-992e-41e0-af95-ba7454891140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:26.761369 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-6b9c0a72-992e-41e0-af95-ba7454891140 None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:26.762981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6b9c0a72-992e-41e0-af95-ba7454891140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:26.861820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 893/1804] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:28:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:26.986762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b410e6d-ed80-4bf4-acd0-88bfc7f29991 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:26.995795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9b410e6d-ed80-4bf4-acd0-88bfc7f29991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:27.126796 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b410e6d-ed80-4bf4-acd0-88bfc7f29991 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:27.133247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c36ef-6d32-4f7d-b250-b33fcd8f6df8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:27.134038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c36ef-6d32-4f7d-b250-b33fcd8f6df8 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:27.134330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c36ef-6d32-4f7d-b250-b33fcd8f6df8 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:27.134799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c36ef-6d32-4f7d-b250-b33fcd8f6df8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:27.145562 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c36ef-6d32-4f7d-b250-b33fcd8f6df8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:27.147239 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c36ef-6d32-4f7d-b250-b33fcd8f6df8 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:27.156341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 912/1805] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:28:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 23:28:27.217353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c0a72-992e-41e0-af95-ba7454891140 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:27.218782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6b9c0a72-992e-41e0-af95-ba7454891140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:27.285289 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c0a72-992e-41e0-af95-ba7454891140 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:27.290055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21df620-8c0d-4c5b-bff7-f0ba8f4b6921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:27.290774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21df620-8c0d-4c5b-bff7-f0ba8f4b6921 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:27.291088 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21df620-8c0d-4c5b-bff7-f0ba8f4b6921 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:27.291319 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21df620-8c0d-4c5b-bff7-f0ba8f4b6921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:27.300996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21df620-8c0d-4c5b-bff7-f0ba8f4b6921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:27.302510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21df620-8c0d-4c5b-bff7-f0ba8f4b6921 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:27.312111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 894/1806] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:37.152588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=46 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:37.307872 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=69,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:38.302915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffaaff-bbad-43d4-b5bd-6944422a05a7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:38.303712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffaaff-bbad-43d4-b5bd-6944422a05a7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:38.303712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffaaff-bbad-43d4-b5bd-6944422a05a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:38.358576 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bdffaaff-bbad-43d4-b5bd-6944422a05a7 None None] MFA Rules not processed for user `bc6e63e9fc3447d5a894a777a98fe773`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:38.364263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bdffaaff-bbad-43d4-b5bd-6944422a05a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:38.486533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 913/1807] 10.222.0.25 () {58 vars in 1015 bytes} [Mon Sep 12 23:28:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:38.521062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21df620-8c0d-4c5b-bff7-f0ba8f4b6921 service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:38.522512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b21df620-8c0d-4c5b-bff7-f0ba8f4b6921 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:38.546659 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21df620-8c0d-4c5b-bff7-f0ba8f4b6921 service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:38.553176 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920bcec-97e5-4aa1-94fa-b1cb026449f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:38.553800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920bcec-97e5-4aa1-94fa-b1cb026449f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:38.554305 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920bcec-97e5-4aa1-94fa-b1cb026449f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:38.554586 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920bcec-97e5-4aa1-94fa-b1cb026449f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:38.586017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920bcec-97e5-4aa1-94fa-b1cb026449f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:38.587475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920bcec-97e5-4aa1-94fa-b1cb026449f9 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:38.603400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 895/1808] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:38.697488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffaaff-bbad-43d4-b5bd-6944422a05a7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:38.788557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffaaff-bbad-43d4-b5bd-6944422a05a7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:38.793413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b46601-7669-4366-b586-f606bcd70943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:38.794050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b46601-7669-4366-b586-f606bcd70943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:38.794362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b46601-7669-4366-b586-f606bcd70943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:38.794639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b46601-7669-4366-b586-f606bcd70943 admin admin] PATH_INFO: `/v3/users/9f5a9cd2a87f480aa8a34b6bd90cbe48` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:38.795867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b46601-7669-4366-b586-f606bcd70943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f5a9cd2a87f480aa8a34b6bd90cbe48)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:38.798078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b46601-7669-4366-b586-f606bcd70943 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:38.854078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-08b46601-7669-4366-b586-f606bcd70943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f5a9cd2a87f480aa8a34b6bd90cbe48'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:38.861623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-08b46601-7669-4366-b586-f606bcd70943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f5a9cd2a87f480aa8a34b6bd90cbe48'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:38.873452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-08b46601-7669-4366-b586-f606bcd70943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f5a9cd2a87f480aa8a34b6bd90cbe48'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:38.882154 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-08b46601-7669-4366-b586-f606bcd70943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f5a9cd2a87f480aa8a34b6bd90cbe48'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:38.882828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-08b46601-7669-4366-b586-f606bcd70943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f5a9cd2a87f480aa8a34b6bd90cbe48'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:38.890348 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 914/1809] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:38 2022] DELETE /identity/v3/users/9f5a9cd2a87f480aa8a34b6bd90cbe48 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:38.895482 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2b67b-55ae-41d0-9676-8fb380b5c5a4 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:38.895861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2b67b-55ae-41d0-9676-8fb380b5c5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:38.896208 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2b67b-55ae-41d0-9676-8fb380b5c5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:38.917340 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-bbc2b67b-55ae-41d0-9676-8fb380b5c5a4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:38.918940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bbc2b67b-55ae-41d0-9676-8fb380b5c5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:39.019331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 896/1810] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:39.035033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b46601-7669-4366-b586-f606bcd70943 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:39.036559 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-08b46601-7669-4366-b586-f606bcd70943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:39.136449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:39.198973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b46601-7669-4366-b586-f606bcd70943 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:39.218762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3483e2-bd96-4ec2-b435-dcbea80e857b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:39.219485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3483e2-bd96-4ec2-b435-dcbea80e857b service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:39.219794 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3483e2-bd96-4ec2-b435-dcbea80e857b service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:39.223259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3483e2-bd96-4ec2-b435-dcbea80e857b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:39.260082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3483e2-bd96-4ec2-b435-dcbea80e857b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:39.261553 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3483e2-bd96-4ec2-b435-dcbea80e857b service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:39.293683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 915/1811] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:40.103909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2b67b-55ae-41d0-9676-8fb380b5c5a4 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:40.107645 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bbc2b67b-55ae-41d0-9676-8fb380b5c5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:40.242323 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2b67b-55ae-41d0-9676-8fb380b5c5a4 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:40.255270 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7536eac-f1a8-48fe-bfae-3a6529f560ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:40.256374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7536eac-f1a8-48fe-bfae-3a6529f560ac service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:40.256918 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7536eac-f1a8-48fe-bfae-3a6529f560ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:40.257337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7536eac-f1a8-48fe-bfae-3a6529f560ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:40.259722 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c7536eac-f1a8-48fe-bfae-3a6529f560ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:40.363590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7536eac-f1a8-48fe-bfae-3a6529f560ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:40.380067 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7536eac-f1a8-48fe-bfae-3a6529f560ac service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:40.393193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3483e2-bd96-4ec2-b435-dcbea80e857b service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:40.395677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eb3483e2-bd96-4ec2-b435-dcbea80e857b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:40.433742 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3483e2-bd96-4ec2-b435-dcbea80e857b service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:40.436517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 897/1812] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:40.449971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:40.454217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:40.456342 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:40.456876 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73 admin admin] PATH_INFO: `/v3/projects/83ff5deecb06424b8abf31397c5af0a0` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:40.461003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83ff5deecb06424b8abf31397c5af0a0)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:40.472898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:40.609310 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83ff5deecb06424b8abf31397c5af0a0', 'request_id': 'req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:40.625046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83ff5deecb06424b8abf31397c5af0a0', 'request_id': 'req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:40.636588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83ff5deecb06424b8abf31397c5af0a0', 'request_id': 'req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:40.661673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73 admin admin] The token cache is being invalidate because project 83ff5deecb06424b8abf31397c5af0a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:40.661950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:40.662675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a819c93-bde7-4d12-b4e5-572ae7cd1c73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:40.665694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 916/1813] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:40 2022] DELETE /identity/v3/projects/83ff5deecb06424b8abf31397c5af0a0 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:28:40.673644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43549699-931b-416b-8263-2e79279c8017 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:40.673922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43549699-931b-416b-8263-2e79279c8017 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:40.675328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43549699-931b-416b-8263-2e79279c8017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:40.732885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-43549699-931b-416b-8263-2e79279c8017 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:40.735239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-43549699-931b-416b-8263-2e79279c8017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:40.809335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:40.872160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10d152-dcc1-42d6-9ead-83a10185ac9e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:40.872483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10d152-dcc1-42d6-9ead-83a10185ac9e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:40.872850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10d152-dcc1-42d6-9ead-83a10185ac9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:40.884055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 898/1814] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:40.885512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38131f6b-0b22-4dbb-8b69-fcd451f314c7 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:40.885800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38131f6b-0b22-4dbb-8b69-fcd451f314c7 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:40.886073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38131f6b-0b22-4dbb-8b69-fcd451f314c7 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:40.887413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 899/1815] 10.222.0.27 () {54 vars in 921 bytes} [Mon Sep 12 23:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:28:40.935707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dc10d152-dcc1-42d6-9ead-83a10185ac9e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:40.938945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dc10d152-dcc1-42d6-9ead-83a10185ac9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:40.972220 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 917/1816] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:28:40.976673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38131f6b-0b22-4dbb-8b69-fcd451f314c7 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:40.995893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38131f6b-0b22-4dbb-8b69-fcd451f314c7 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:41.005860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc8daa-df5d-4c33-987f-fa05f3a24f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:41.006734 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc8daa-df5d-4c33-987f-fa05f3a24f3a admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:41.007021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc8daa-df5d-4c33-987f-fa05f3a24f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:41.007359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc8daa-df5d-4c33-987f-fa05f3a24f3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:41.008149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc8daa-df5d-4c33-987f-fa05f3a24f3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:28:41.008521 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc8daa-df5d-4c33-987f-fa05f3a24f3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:41.010472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc8daa-df5d-4c33-987f-fa05f3a24f3a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:41.073901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 900/1817] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:28:41.078968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10d152-dcc1-42d6-9ead-83a10185ac9e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:41.091159 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10d152-dcc1-42d6-9ead-83a10185ac9e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:41.107227 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bc80a-4256-4774-ad56-9a4cf0daadc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:41.108965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bc80a-4256-4774-ad56-9a4cf0daadc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:41.108965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bc80a-4256-4774-ad56-9a4cf0daadc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:41.108965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bc80a-4256-4774-ad56-9a4cf0daadc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:41.109551 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89bc80a-4256-4774-ad56-9a4cf0daadc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:41.111620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a0d3c-c38b-48b0-b276-d5cff68dca98 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:41.112518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a0d3c-c38b-48b0-b276-d5cff68dca98 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:41.112805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89bc80a-4256-4774-ad56-9a4cf0daadc8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:41.113605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a0d3c-c38b-48b0-b276-d5cff68dca98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:41.163029 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-3a2a0d3c-c38b-48b0-b276-d5cff68dca98 None None] MFA Rules not processed for user `f5fa28ffdafd435cb2d93bd453add3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:41.165598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3a2a0d3c-c38b-48b0-b276-d5cff68dca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:41.187777 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 918/1818] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:41 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:41.192166 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bc80a-4256-4774-ad56-9a4cf0daadc8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:41.273550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bc80a-4256-4774-ad56-9a4cf0daadc8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:41.281419 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed381564-6bf4-4561-83ee-73f5bf4f52f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:41.282311 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed381564-6bf4-4561-83ee-73f5bf4f52f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:41.282660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed381564-6bf4-4561-83ee-73f5bf4f52f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:41.283115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed381564-6bf4-4561-83ee-73f5bf4f52f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:41.283886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed381564-6bf4-4561-83ee-73f5bf4f52f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:41.286367 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed381564-6bf4-4561-83ee-73f5bf4f52f5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:41.318357 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 919/1819] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:41 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:28:41.319771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed381564-6bf4-4561-83ee-73f5bf4f52f5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:41.325502 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 901/1820] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Sep 12 23:28:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:41.332356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed381564-6bf4-4561-83ee-73f5bf4f52f5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:41.341526 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480bd5e-7085-4082-85b7-443d895013ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:41.342340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480bd5e-7085-4082-85b7-443d895013ac admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:41.342771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480bd5e-7085-4082-85b7-443d895013ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:41.343709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480bd5e-7085-4082-85b7-443d895013ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:41.345214 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480bd5e-7085-4082-85b7-443d895013ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:41.346962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480bd5e-7085-4082-85b7-443d895013ac admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:41.371341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 920/1821] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:41.375706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a0d3c-c38b-48b0-b276-d5cff68dca98 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:41.392138 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a0d3c-c38b-48b0-b276-d5cff68dca98 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:41.399492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea65ecb-95f7-4cba-8421-ee6c251c0716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:41.400310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea65ecb-95f7-4cba-8421-ee6c251c0716 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:41.400667 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea65ecb-95f7-4cba-8421-ee6c251c0716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:41.401038 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea65ecb-95f7-4cba-8421-ee6c251c0716 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:41.401891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea65ecb-95f7-4cba-8421-ee6c251c0716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:41.403999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea65ecb-95f7-4cba-8421-ee6c251c0716 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:41.419484 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 902/1822] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:41.424094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480bd5e-7085-4082-85b7-443d895013ac admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:41.442030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:41.445230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea65ecb-95f7-4cba-8421-ee6c251c0716 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:41.451088 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-eea65ecb-95f7-4cba-8421-ee6c251c0716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:41.455888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480bd5e-7085-4082-85b7-443d895013ac admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:41.466912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027c5b4-36aa-4d3f-95cd-039a22fab003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:41.471679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027c5b4-36aa-4d3f-95cd-039a22fab003 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:41.471968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027c5b4-36aa-4d3f-95cd-039a22fab003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:41.472249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027c5b4-36aa-4d3f-95cd-039a22fab003 admin admin] PATH_INFO: `/v3/projects/641b6882f873400fbef301fc5ce82391/users/2dcd3dd65d5841049ed20e36e847bb53/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:41.510732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027c5b4-36aa-4d3f-95cd-039a22fab003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=641b6882f873400fbef301fc5ce82391, user_id=2dcd3dd65d5841049ed20e36e847bb53, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:41.519464 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027c5b4-36aa-4d3f-95cd-039a22fab003 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:41.573977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 921/1823] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:41 2022] PUT /identity/v3/projects/641b6882f873400fbef301fc5ce82391/users/2dcd3dd65d5841049ed20e36e847bb53/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:41.597722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae5ac2-d976-4610-a3a3-7872bdafc46d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:41.598026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae5ac2-d976-4610-a3a3-7872bdafc46d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:41.598322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae5ac2-d976-4610-a3a3-7872bdafc46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:41.646152 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-fbae5ac2-d976-4610-a3a3-7872bdafc46d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:41.646712 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea65ecb-95f7-4cba-8421-ee6c251c0716 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:41.653985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fbae5ac2-d976-4610-a3a3-7872bdafc46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:41.808484 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:41.857713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 922/1824] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:28:41.865882 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd884c47-fe5e-45e6-911e-74c74d620d99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:41.867036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd884c47-fe5e-45e6-911e-74c74d620d99 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:41.868268 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd884c47-fe5e-45e6-911e-74c74d620d99 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:41.869144 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd884c47-fe5e-45e6-911e-74c74d620d99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:41.877816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae5ac2-d976-4610-a3a3-7872bdafc46d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:41.879251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fbae5ac2-d976-4610-a3a3-7872bdafc46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:41.893267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd884c47-fe5e-45e6-911e-74c74d620d99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:41.902275 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd884c47-fe5e-45e6-911e-74c74d620d99 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:41.929654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 903/1825] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:42.075486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae5ac2-d976-4610-a3a3-7872bdafc46d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:42.084022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d813c-ed2f-4c75-9cb3-48d1f768cb6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:42.084661 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d813c-ed2f-4c75-9cb3-48d1f768cb6d service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:42.084941 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d813c-ed2f-4c75-9cb3-48d1f768cb6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:42.085212 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d813c-ed2f-4c75-9cb3-48d1f768cb6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:42.100142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d813c-ed2f-4c75-9cb3-48d1f768cb6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:42.101916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d813c-ed2f-4c75-9cb3-48d1f768cb6d service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:42.122972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 923/1826] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:28:42.180695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd884c47-fe5e-45e6-911e-74c74d620d99 service nova] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:42.182179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-fd884c47-fe5e-45e6-911e-74c74d620d99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:42.211175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd884c47-fe5e-45e6-911e-74c74d620d99 service nova] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:42.216817 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0206a-f7de-4254-97c9-e8926d099b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:42.217757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0206a-f7de-4254-97c9-e8926d099b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:42.219443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0206a-f7de-4254-97c9-e8926d099b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:42.220014 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0206a-f7de-4254-97c9-e8926d099b22 admin admin] PATH_INFO: `/v3/users/34c651a6a1a140eebf4914f521756552` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:42.221450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0206a-f7de-4254-97c9-e8926d099b22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34c651a6a1a140eebf4914f521756552)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:42.223516 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0206a-f7de-4254-97c9-e8926d099b22 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:42.278818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-12b0206a-f7de-4254-97c9-e8926d099b22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34c651a6a1a140eebf4914f521756552'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:42.292453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-12b0206a-f7de-4254-97c9-e8926d099b22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34c651a6a1a140eebf4914f521756552'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:42.301428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-12b0206a-f7de-4254-97c9-e8926d099b22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34c651a6a1a140eebf4914f521756552'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:42.302230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-12b0206a-f7de-4254-97c9-e8926d099b22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34c651a6a1a140eebf4914f521756552'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:42.311079 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-12b0206a-f7de-4254-97c9-e8926d099b22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34c651a6a1a140eebf4914f521756552'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:42.318510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 904/1827] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:42 2022] DELETE /identity/v3/users/34c651a6a1a140eebf4914f521756552 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:42.325185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598e6d4-2555-4e0e-8709-a35940d4d4c5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:42.325886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598e6d4-2555-4e0e-8709-a35940d4d4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:42.326325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598e6d4-2555-4e0e-8709-a35940d4d4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:42.358957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4598e6d4-2555-4e0e-8709-a35940d4d4c5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:42.360768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4598e6d4-2555-4e0e-8709-a35940d4d4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:42.462486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 924/1828] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:42.487783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0206a-f7de-4254-97c9-e8926d099b22 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:42.489406 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-12b0206a-f7de-4254-97c9-e8926d099b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:42.560879 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0206a-f7de-4254-97c9-e8926d099b22 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:42.565384 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9e9b4-1321-4ed9-95de-9c14527cbe8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:42.565997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9e9b4-1321-4ed9-95de-9c14527cbe8f service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:42.566277 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9e9b4-1321-4ed9-95de-9c14527cbe8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:42.566593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9e9b4-1321-4ed9-95de-9c14527cbe8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:42.579824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9e9b4-1321-4ed9-95de-9c14527cbe8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:42.581282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9e9b4-1321-4ed9-95de-9c14527cbe8f service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:42.598163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 905/1829] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:43.418001 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bb13e-2e43-4702-a4be-a68f1411e246 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:43.418551 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bb13e-2e43-4702-a4be-a68f1411e246 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:43.418899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bb13e-2e43-4702-a4be-a68f1411e246 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:43.421544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 925/1830] 10.222.0.27 () {54 vars in 921 bytes} [Mon Sep 12 23:28:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:28:43.450530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382a359-4db8-43a5-80b8-17cde9e68623 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:43.452131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382a359-4db8-43a5-80b8-17cde9e68623 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:43.453068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382a359-4db8-43a5-80b8-17cde9e68623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:43.519955 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-1382a359-4db8-43a5-80b8-17cde9e68623 None None] MFA Rules not processed for user `f5fa28ffdafd435cb2d93bd453add3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:43.523406 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1382a359-4db8-43a5-80b8-17cde9e68623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:43.619204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:43.669506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 906/1831] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Sep 12 23:28:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:43.871711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850bb13e-2e43-4702-a4be-a68f1411e246 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:43.874934 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-850bb13e-2e43-4702-a4be-a68f1411e246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:43.915381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850bb13e-2e43-4702-a4be-a68f1411e246 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:43.917967 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382a359-4db8-43a5-80b8-17cde9e68623 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:43.920597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1382a359-4db8-43a5-80b8-17cde9e68623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:43.937819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafee3eb-5244-4ecc-b42d-4938aac364c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:43.938549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee3eb-5244-4ecc-b42d-4938aac364c0 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:43.938944 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee3eb-5244-4ecc-b42d-4938aac364c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:43.939290 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee3eb-5244-4ecc-b42d-4938aac364c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:43.940737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382a359-4db8-43a5-80b8-17cde9e68623 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:43.946163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755dee3a-a189-4e55-898f-0421c43dd4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:43.947056 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755dee3a-a189-4e55-898f-0421c43dd4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:43.947515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755dee3a-a189-4e55-898f-0421c43dd4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:43.948016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755dee3a-a189-4e55-898f-0421c43dd4d8 admin admin] PATH_INFO: `/v3/users/7be0f2d4f7344ccba50c3e86f478e568` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:43.950297 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755dee3a-a189-4e55-898f-0421c43dd4d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be0f2d4f7344ccba50c3e86f478e568)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:43.952613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755dee3a-a189-4e55-898f-0421c43dd4d8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:43.966757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafee3eb-5244-4ecc-b42d-4938aac364c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:43.968678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafee3eb-5244-4ecc-b42d-4938aac364c0 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:43.997545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 926/1832] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:44.006638 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafee3eb-5244-4ecc-b42d-4938aac364c0 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:44.008388 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dafee3eb-5244-4ecc-b42d-4938aac364c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:44.034876 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafee3eb-5244-4ecc-b42d-4938aac364c0 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:44.044251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:44.046167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:44.046692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:44.047651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] PATH_INFO: `/v3/projects/1f606e6b59bd48daad82fdf38e01ae4d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:44.049284 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f606e6b59bd48daad82fdf38e01ae4d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:44.052549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:44.064765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:44.103236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-755dee3a-a189-4e55-898f-0421c43dd4d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be0f2d4f7344ccba50c3e86f478e568'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.126239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-755dee3a-a189-4e55-898f-0421c43dd4d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be0f2d4f7344ccba50c3e86f478e568'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.143510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-755dee3a-a189-4e55-898f-0421c43dd4d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be0f2d4f7344ccba50c3e86f478e568'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.144532 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-755dee3a-a189-4e55-898f-0421c43dd4d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be0f2d4f7344ccba50c3e86f478e568'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.150994 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f606e6b59bd48daad82fdf38e01ae4d', 'request_id': 'req-6775469e-d103-4e59-8b91-98e46fce933d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.155987 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-755dee3a-a189-4e55-898f-0421c43dd4d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be0f2d4f7344ccba50c3e86f478e568'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.161018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f606e6b59bd48daad82fdf38e01ae4d', 'request_id': 'req-6775469e-d103-4e59-8b91-98e46fce933d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.167540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 907/1833] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:43 2022] DELETE /identity/v3/users/7be0f2d4f7344ccba50c3e86f478e568 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:44.174149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7ea8-0e53-4bea-9d58-49f500379c2f None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:44.174528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7ea8-0e53-4bea-9d58-49f500379c2f None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:44.174833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7ea8-0e53-4bea-9d58-49f500379c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:44.177362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f606e6b59bd48daad82fdf38e01ae4d', 'request_id': 'req-6775469e-d103-4e59-8b91-98e46fce933d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.196211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] The token cache is being invalidate because project 1f606e6b59bd48daad82fdf38e01ae4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:44.196540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.197494 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.199676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 927/1834] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:44 2022] DELETE /identity/v3/projects/1f606e6b59bd48daad82fdf38e01ae4d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:44.216149 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:44.216470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-627e7ea8-0e53-4bea-9d58-49f500379c2f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:44.216470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-627e7ea8-0e53-4bea-9d58-49f500379c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:44.218072 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:44.399935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775469e-d103-4e59-8b91-98e46fce933d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:44.441389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 908/1835] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:44.442834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a462043-8a92-4670-b210-67b73a57f8c9 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:44.443120 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a462043-8a92-4670-b210-67b73a57f8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:44.443394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a462043-8a92-4670-b210-67b73a57f8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:44.504155 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-1a462043-8a92-4670-b210-67b73a57f8c9 None None] MFA Rules not processed for user `2dcd3dd65d5841049ed20e36e847bb53`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:44.506028 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1a462043-8a92-4670-b210-67b73a57f8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:44.518609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798646ef-bcf7-4854-97de-b26dfdda9e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:44.521852 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798646ef-bcf7-4854-97de-b26dfdda9e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:44.522220 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798646ef-bcf7-4854-97de-b26dfdda9e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:44.522566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798646ef-bcf7-4854-97de-b26dfdda9e23 admin admin] PATH_INFO: `/v3/users/0fcabc07bca14f80a7329d876a35e30b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:44.524004 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798646ef-bcf7-4854-97de-b26dfdda9e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fcabc07bca14f80a7329d876a35e30b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:44.527263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798646ef-bcf7-4854-97de-b26dfdda9e23 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:44.589893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:44.615954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:44.642716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 909/1836] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:44.642716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a462043-8a92-4670-b210-67b73a57f8c9 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:44.642716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1a462043-8a92-4670-b210-67b73a57f8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:44.648839 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-798646ef-bcf7-4854-97de-b26dfdda9e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fcabc07bca14f80a7329d876a35e30b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.667772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-798646ef-bcf7-4854-97de-b26dfdda9e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fcabc07bca14f80a7329d876a35e30b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.694851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-798646ef-bcf7-4854-97de-b26dfdda9e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fcabc07bca14f80a7329d876a35e30b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.710179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-798646ef-bcf7-4854-97de-b26dfdda9e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fcabc07bca14f80a7329d876a35e30b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.710846 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-798646ef-bcf7-4854-97de-b26dfdda9e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fcabc07bca14f80a7329d876a35e30b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:44.721549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 928/1837] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:44 2022] DELETE /identity/v3/users/0fcabc07bca14f80a7329d876a35e30b => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:44.722509 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798646ef-bcf7-4854-97de-b26dfdda9e23 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:44.724918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-798646ef-bcf7-4854-97de-b26dfdda9e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:44.767324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a462043-8a92-4670-b210-67b73a57f8c9 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:44.847711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798646ef-bcf7-4854-97de-b26dfdda9e23 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:44.855800 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9e860-6658-474f-903f-e94005082a9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:44.856427 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9e860-6658-474f-903f-e94005082a9c service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:44.857186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9e860-6658-474f-903f-e94005082a9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:44.857484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9e860-6658-474f-903f-e94005082a9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:44.859644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ccd9e860-6658-474f-903f-e94005082a9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:44.874851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e9295-1010-402c-be6c-8486af71f8c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:44.876071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e9295-1010-402c-be6c-8486af71f8c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:44.876356 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e9295-1010-402c-be6c-8486af71f8c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:44.876630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e9295-1010-402c-be6c-8486af71f8c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:44.881293 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-764e9295-1010-402c-be6c-8486af71f8c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:44.974736 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9e860-6658-474f-903f-e94005082a9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:44.977644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9e860-6658-474f-903f-e94005082a9c service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:44.983672 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e9295-1010-402c-be6c-8486af71f8c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:44.985540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e9295-1010-402c-be6c-8486af71f8c6 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:44.987260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 929/1838] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:44.998965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 910/1839] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:46.094023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9e860-6658-474f-903f-e94005082a9c service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:46.095841 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ccd9e860-6658-474f-903f-e94005082a9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:46.115335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9e860-6658-474f-903f-e94005082a9c service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:46.127705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67255f69-2460-4d0e-9f44-3419262b64ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:46.129078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67255f69-2460-4d0e-9f44-3419262b64ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:46.129767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67255f69-2460-4d0e-9f44-3419262b64ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:46.130524 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67255f69-2460-4d0e-9f44-3419262b64ca admin admin] PATH_INFO: `/v3/projects/23fc656bfad747739691f0b18e91e331` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:46.134290 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67255f69-2460-4d0e-9f44-3419262b64ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=23fc656bfad747739691f0b18e91e331)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:46.139805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67255f69-2460-4d0e-9f44-3419262b64ca admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:46.156564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e9295-1010-402c-be6c-8486af71f8c6 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:46.158751 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-764e9295-1010-402c-be6c-8486af71f8c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:46.193291 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e9295-1010-402c-be6c-8486af71f8c6 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:46.228637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:46.229365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:46.230534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:46.230831 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] PATH_INFO: `/v3/projects/9f79837133d94ceea715cf1c44716924` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:46.233258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f79837133d94ceea715cf1c44716924)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:46.238352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:46.313844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-67255f69-2460-4d0e-9f44-3419262b64ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23fc656bfad747739691f0b18e91e331', 'request_id': 'req-67255f69-2460-4d0e-9f44-3419262b64ca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.333291 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-67255f69-2460-4d0e-9f44-3419262b64ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23fc656bfad747739691f0b18e91e331', 'request_id': 'req-67255f69-2460-4d0e-9f44-3419262b64ca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.348037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-67255f69-2460-4d0e-9f44-3419262b64ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23fc656bfad747739691f0b18e91e331', 'request_id': 'req-67255f69-2460-4d0e-9f44-3419262b64ca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.355105 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:46.361699 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:46.363425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f79837133d94ceea715cf1c44716924', 'request_id': 'req-e5279573-5787-46f1-aaea-1b2869bf4ec9'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.375072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f79837133d94ceea715cf1c44716924', 'request_id': 'req-e5279573-5787-46f1-aaea-1b2869bf4ec9'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.377088 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-67255f69-2460-4d0e-9f44-3419262b64ca admin admin] The token cache is being invalidate because project 23fc656bfad747739691f0b18e91e331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:46.377399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-67255f69-2460-4d0e-9f44-3419262b64ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.378202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-67255f69-2460-4d0e-9f44-3419262b64ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.384349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 930/1840] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:46 2022] DELETE /identity/v3/projects/23fc656bfad747739691f0b18e91e331 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:46.386216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35d1d4-fa38-44ef-90d4-6eefc2958929 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:46.386590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35d1d4-fa38-44ef-90d4-6eefc2958929 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:46.386861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35d1d4-fa38-44ef-90d4-6eefc2958929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:46.392396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f79837133d94ceea715cf1c44716924', 'request_id': 'req-e5279573-5787-46f1-aaea-1b2869bf4ec9'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.409745 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bc35d1d4-fa38-44ef-90d4-6eefc2958929 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:46.410982 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bc35d1d4-fa38-44ef-90d4-6eefc2958929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:46.415254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] The token cache is being invalidate because project 9f79837133d94ceea715cf1c44716924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:46.415779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.416689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.418217 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 911/1841] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:46 2022] DELETE /identity/v3/projects/9f79837133d94ceea715cf1c44716924 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:46.422615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:46.436706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:46.485660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 931/1842] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:46.496258 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961373c-096d-40fb-b712-3fefe9057b80 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:46.496539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961373c-096d-40fb-b712-3fefe9057b80 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:46.498261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961373c-096d-40fb-b712-3fefe9057b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:46.533258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5279573-5787-46f1-aaea-1b2869bf4ec9 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:46.538520 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0961373c-096d-40fb-b712-3fefe9057b80 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:46.539801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0961373c-096d-40fb-b712-3fefe9057b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:46.541440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477223a6-656b-47d4-89db-330387e82e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:46.542493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477223a6-656b-47d4-89db-330387e82e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:46.543471 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477223a6-656b-47d4-89db-330387e82e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:46.544273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477223a6-656b-47d4-89db-330387e82e42 admin admin] PATH_INFO: `/v3/users/b763d6c5c2c545f69a7106a9162c6a23` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:46.546075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477223a6-656b-47d4-89db-330387e82e42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b763d6c5c2c545f69a7106a9162c6a23)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:46.550437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477223a6-656b-47d4-89db-330387e82e42 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:46.553437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 932/1843] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:46.554518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961373c-096d-40fb-b712-3fefe9057b80 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:46.566611 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961373c-096d-40fb-b712-3fefe9057b80 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:46.572002 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0d05d-befe-4d7b-942f-01a0d192edfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:46.572642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0d05d-befe-4d7b-942f-01a0d192edfd admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:46.572929 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0d05d-befe-4d7b-942f-01a0d192edfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:46.573207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0d05d-befe-4d7b-942f-01a0d192edfd admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:46.574193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0d05d-befe-4d7b-942f-01a0d192edfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:28:46.574642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0d05d-befe-4d7b-942f-01a0d192edfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:46.576393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0d05d-befe-4d7b-942f-01a0d192edfd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:46.607012 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 933/1844] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:28:46.608117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0d05d-befe-4d7b-942f-01a0d192edfd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:46.649692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0d05d-befe-4d7b-942f-01a0d192edfd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:46.652779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-477223a6-656b-47d4-89db-330387e82e42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b763d6c5c2c545f69a7106a9162c6a23'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.658916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9596b2-0253-4763-bada-2644ca3173a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:46.659986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9596b2-0253-4763-bada-2644ca3173a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:46.660281 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9596b2-0253-4763-bada-2644ca3173a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:46.660564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9596b2-0253-4763-bada-2644ca3173a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:46.661369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9596b2-0253-4763-bada-2644ca3173a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:46.664125 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9596b2-0253-4763-bada-2644ca3173a8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:46.682686 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-477223a6-656b-47d4-89db-330387e82e42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b763d6c5c2c545f69a7106a9162c6a23'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.699116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-477223a6-656b-47d4-89db-330387e82e42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b763d6c5c2c545f69a7106a9162c6a23'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.700774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-477223a6-656b-47d4-89db-330387e82e42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b763d6c5c2c545f69a7106a9162c6a23'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.713723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-477223a6-656b-47d4-89db-330387e82e42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b763d6c5c2c545f69a7106a9162c6a23'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:46.725072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 912/1845] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:46 2022] DELETE /identity/v3/users/b763d6c5c2c545f69a7106a9162c6a23 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:46.726193 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff4863-5b5f-439a-aef8-67f0f9633a2c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:46.726517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff4863-5b5f-439a-aef8-67f0f9633a2c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:46.726939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff4863-5b5f-439a-aef8-67f0f9633a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:46.747358 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 934/1846] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:46 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:46.754564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9596b2-0253-4763-bada-2644ca3173a8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:46.760463 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7b9596b2-0253-4763-bada-2644ca3173a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:46.779198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-94ff4863-5b5f-439a-aef8-67f0f9633a2c None None] MFA Rules not processed for user `2dcd3dd65d5841049ed20e36e847bb53`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:46.781107 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-94ff4863-5b5f-439a-aef8-67f0f9633a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:46.840710 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:46.855786 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 913/1847] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:46.942762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9596b2-0253-4763-bada-2644ca3173a8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:46.948304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b68921-f1ad-489c-9d3f-a82216246155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:46.949333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b68921-f1ad-489c-9d3f-a82216246155 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:46.950061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b68921-f1ad-489c-9d3f-a82216246155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:46.950250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b68921-f1ad-489c-9d3f-a82216246155 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:46.951929 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b68921-f1ad-489c-9d3f-a82216246155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:46.955038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b68921-f1ad-489c-9d3f-a82216246155 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:46.987642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 935/1848] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:46 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:46.991720 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff4863-5b5f-439a-aef8-67f0f9633a2c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:47.007481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff4863-5b5f-439a-aef8-67f0f9633a2c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:47.014241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed2770-1ae0-48fb-bfa9-2d3471e4518f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:47.014241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed2770-1ae0-48fb-bfa9-2d3471e4518f admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:47.014241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed2770-1ae0-48fb-bfa9-2d3471e4518f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:47.014241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed2770-1ae0-48fb-bfa9-2d3471e4518f admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:47.015536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed2770-1ae0-48fb-bfa9-2d3471e4518f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:47.017697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed2770-1ae0-48fb-bfa9-2d3471e4518f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:47.032766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 914/1849] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:47.036628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b68921-f1ad-489c-9d3f-a82216246155 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:47.050720 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b68921-f1ad-489c-9d3f-a82216246155 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:47.062224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21405a87-e545-41d5-99aa-c636be3c6911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:47.063609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21405a87-e545-41d5-99aa-c636be3c6911 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:47.063999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21405a87-e545-41d5-99aa-c636be3c6911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:47.065132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21405a87-e545-41d5-99aa-c636be3c6911 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:47.065585 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21405a87-e545-41d5-99aa-c636be3c6911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:47.067376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21405a87-e545-41d5-99aa-c636be3c6911 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:47.088617 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 936/1850] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:47.090622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed2770-1ae0-48fb-bfa9-2d3471e4518f admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:47.104368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed2770-1ae0-48fb-bfa9-2d3471e4518f admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:47.111062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d45c0-3836-4261-854c-7d4611ca62f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:47.112564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d45c0-3836-4261-854c-7d4611ca62f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:47.113121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d45c0-3836-4261-854c-7d4611ca62f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:47.113545 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d45c0-3836-4261-854c-7d4611ca62f1 admin admin] PATH_INFO: `/v3/projects/2258ff812cc848118d33c317c470cf4d/users/00019055425b40808a3899a2ec2dd5a5/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:47.132020 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d45c0-3836-4261-854c-7d4611ca62f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2258ff812cc848118d33c317c470cf4d, user_id=00019055425b40808a3899a2ec2dd5a5, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:47.135280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d45c0-3836-4261-854c-7d4611ca62f1 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:47.151339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 915/1851] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:47 2022] PUT /identity/v3/projects/2258ff812cc848118d33c317c470cf4d/users/00019055425b40808a3899a2ec2dd5a5/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:47.162291 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998a2a4-497f-4276-b4fa-d77671be57ed None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:47.163117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998a2a4-497f-4276-b4fa-d77671be57ed None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:47.163592 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998a2a4-497f-4276-b4fa-d77671be57ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:47.200758 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c998a2a4-497f-4276-b4fa-d77671be57ed None None] MFA Rules not processed for user `00019055425b40808a3899a2ec2dd5a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:47.201874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c998a2a4-497f-4276-b4fa-d77671be57ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:47.240298 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:47.295657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 937/1852] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:47.311631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d45c0-3836-4261-854c-7d4611ca62f1 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:47.314118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c93d45c0-3836-4261-854c-7d4611ca62f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:47.330840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998a2a4-497f-4276-b4fa-d77671be57ed None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:47.332292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c998a2a4-497f-4276-b4fa-d77671be57ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:47.397898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d45c0-3836-4261-854c-7d4611ca62f1 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:47.404191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e0724-fde3-45cf-af55-c91dbab732b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:47.404798 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e0724-fde3-45cf-af55-c91dbab732b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:47.405039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e0724-fde3-45cf-af55-c91dbab732b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:47.405744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e0724-fde3-45cf-af55-c91dbab732b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:47.417489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e0724-fde3-45cf-af55-c91dbab732b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:47.418998 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e0724-fde3-45cf-af55-c91dbab732b5 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:47.431463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 916/1853] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:47.461087 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e0724-fde3-45cf-af55-c91dbab732b5 service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:47.482364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998a2a4-497f-4276-b4fa-d77671be57ed None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:47.490197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:47.491209 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:47.492026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:47.492452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] PATH_INFO: `/v3/projects/c2e2315683c440c4b2d4e6a981c2b054` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:47.494692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e2315683c440c4b2d4e6a981c2b054)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:47.497584 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:47.567260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e2315683c440c4b2d4e6a981c2b054', 'request_id': 'req-fbcd6dc8-d708-4aae-9e57-b65b1336397c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:47.573937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e2315683c440c4b2d4e6a981c2b054', 'request_id': 'req-fbcd6dc8-d708-4aae-9e57-b65b1336397c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:47.584380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e2315683c440c4b2d4e6a981c2b054', 'request_id': 'req-fbcd6dc8-d708-4aae-9e57-b65b1336397c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:47.608390 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e0724-fde3-45cf-af55-c91dbab732b5 service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:47.618117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] The token cache is being invalidate because project c2e2315683c440c4b2d4e6a981c2b054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:47.618696 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:47.620582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:47.622458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 938/1854] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:47 2022] DELETE /identity/v3/projects/c2e2315683c440c4b2d4e6a981c2b054 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:28:47.628300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:47.632057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:47.683493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83e110-c367-4d14-b4fc-8e1c137274e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:47.684184 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83e110-c367-4d14-b4fc-8e1c137274e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:47.684462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83e110-c367-4d14-b4fc-8e1c137274e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:47.685271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83e110-c367-4d14-b4fc-8e1c137274e2 admin admin] PATH_INFO: `/v3/users/00019055425b40808a3899a2ec2dd5a5` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:47.686777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83e110-c367-4d14-b4fc-8e1c137274e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00019055425b40808a3899a2ec2dd5a5)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:47.690581 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83e110-c367-4d14-b4fc-8e1c137274e2 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:47.706835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:47.773635 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6c83e110-c367-4d14-b4fc-8e1c137274e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00019055425b40808a3899a2ec2dd5a5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:47.809117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6c83e110-c367-4d14-b4fc-8e1c137274e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00019055425b40808a3899a2ec2dd5a5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:47.816541 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:47.820814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6c83e110-c367-4d14-b4fc-8e1c137274e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00019055425b40808a3899a2ec2dd5a5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:47.821619 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6c83e110-c367-4d14-b4fc-8e1c137274e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00019055425b40808a3899a2ec2dd5a5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:47.827847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd6dc8-d708-4aae-9e57-b65b1336397c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:47.840260 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6c83e110-c367-4d14-b4fc-8e1c137274e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00019055425b40808a3899a2ec2dd5a5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:47.841405 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a846f1e-f2ca-4378-a09e-b10ec5234044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:47.842069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a846f1e-f2ca-4378-a09e-b10ec5234044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:47.842351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a846f1e-f2ca-4378-a09e-b10ec5234044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:47.842628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a846f1e-f2ca-4378-a09e-b10ec5234044 admin admin] PATH_INFO: `/v3/users/62137425104c4acf8251bc34aa870c79` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:47.849452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a846f1e-f2ca-4378-a09e-b10ec5234044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62137425104c4acf8251bc34aa870c79)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:47.852006 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a846f1e-f2ca-4378-a09e-b10ec5234044 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:47.861012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 917/1855] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:47 2022] DELETE /identity/v3/users/00019055425b40808a3899a2ec2dd5a5 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:47.862568 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d335c6-fde9-4956-9e66-8a01e5949a13 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:47.862863 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d335c6-fde9-4956-9e66-8a01e5949a13 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:47.864238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d335c6-fde9-4956-9e66-8a01e5949a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:47.931694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-57d335c6-fde9-4956-9e66-8a01e5949a13 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:47.933147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-57d335c6-fde9-4956-9e66-8a01e5949a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:47.999019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a846f1e-f2ca-4378-a09e-b10ec5234044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62137425104c4acf8251bc34aa870c79'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:48.010247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a846f1e-f2ca-4378-a09e-b10ec5234044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62137425104c4acf8251bc34aa870c79'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:48.055118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a846f1e-f2ca-4378-a09e-b10ec5234044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62137425104c4acf8251bc34aa870c79'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:48.072056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a846f1e-f2ca-4378-a09e-b10ec5234044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62137425104c4acf8251bc34aa870c79'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:48.073326 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a846f1e-f2ca-4378-a09e-b10ec5234044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62137425104c4acf8251bc34aa870c79'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:48.083613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 918/1856] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:48.086034 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326d864-4506-4b0f-8ac8-394522f1f976 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:48.086504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326d864-4506-4b0f-8ac8-394522f1f976 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:48.087137 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326d864-4506-4b0f-8ac8-394522f1f976 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:48.089295 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 939/1857] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:28:47 2022] DELETE /identity/v3/users/62137425104c4acf8251bc34aa870c79 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:48.090089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 919/1858] 10.222.0.27 () {54 vars in 921 bytes} [Mon Sep 12 23:28:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:28:48.095291 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a846f1e-f2ca-4378-a09e-b10ec5234044 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:48.099679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1a846f1e-f2ca-4378-a09e-b10ec5234044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:48.108626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc67406-6843-4cd7-a8e6-ef2ea70d4199 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:48.109442 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc67406-6843-4cd7-a8e6-ef2ea70d4199 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:48.110015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc67406-6843-4cd7-a8e6-ef2ea70d4199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:48.136338 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-7dc67406-6843-4cd7-a8e6-ef2ea70d4199 None None] MFA Rules not processed for user `f5fa28ffdafd435cb2d93bd453add3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:48.137869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7dc67406-6843-4cd7-a8e6-ef2ea70d4199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:48.214833 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a846f1e-f2ca-4378-a09e-b10ec5234044 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:48.230017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c0606-bf76-4661-a492-b8feb5a15b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:48.237034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c0606-bf76-4661-a492-b8feb5a15b58 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:48.238047 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c0606-bf76-4661-a492-b8feb5a15b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:48.247316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c0606-bf76-4661-a492-b8feb5a15b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:48.355274 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 920/1859] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Sep 12 23:28:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:48.409951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c0606-bf76-4661-a492-b8feb5a15b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:48.411888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c0606-bf76-4661-a492-b8feb5a15b58 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:48.436675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 940/1860] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:48.621258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc67406-6843-4cd7-a8e6-ef2ea70d4199 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:48.623203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7dc67406-6843-4cd7-a8e6-ef2ea70d4199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:48.637890 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc67406-6843-4cd7-a8e6-ef2ea70d4199 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:48.647002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520a5de-c215-40a3-b3b9-f6379be8f7de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:48.647688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520a5de-c215-40a3-b3b9-f6379be8f7de service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:48.647974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520a5de-c215-40a3-b3b9-f6379be8f7de service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:48.648258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520a5de-c215-40a3-b3b9-f6379be8f7de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:48.663165 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520a5de-c215-40a3-b3b9-f6379be8f7de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:48.664832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520a5de-c215-40a3-b3b9-f6379be8f7de service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:48.678420 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 921/1861] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:49.428156 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c0606-bf76-4661-a492-b8feb5a15b58 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:49.429584 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a37c0606-bf76-4661-a492-b8feb5a15b58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:49.442313 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c0606-bf76-4661-a492-b8feb5a15b58 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:49.448430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:49.449512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:49.449973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:49.450456 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991 admin admin] PATH_INFO: `/v3/projects/daa2a66bfcbd41e6aa49510abf1f4b9e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:49.451864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa2a66bfcbd41e6aa49510abf1f4b9e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:49.454848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:49.479359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520a5de-c215-40a3-b3b9-f6379be8f7de service nova] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:49.482083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7520a5de-c215-40a3-b3b9-f6379be8f7de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:49.488945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:49.507875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520a5de-c215-40a3-b3b9-f6379be8f7de service nova] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:49.516558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e55f8-5710-4343-8495-67ac7a2e46be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:49.518165 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e55f8-5710-4343-8495-67ac7a2e46be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:49.519938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e55f8-5710-4343-8495-67ac7a2e46be admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:49.520405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e55f8-5710-4343-8495-67ac7a2e46be admin admin] PATH_INFO: `/v3/projects/2258ff812cc848118d33c317c470cf4d` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:49.522448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e55f8-5710-4343-8495-67ac7a2e46be admin admin] RBAC: Authorizing `identity:delete_project(project_id=2258ff812cc848118d33c317c470cf4d)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:49.525010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e55f8-5710-4343-8495-67ac7a2e46be admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:49.532693 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:49.542007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa2a66bfcbd41e6aa49510abf1f4b9e', 'request_id': 'req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:49.555157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa2a66bfcbd41e6aa49510abf1f4b9e', 'request_id': 'req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:49.571621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa2a66bfcbd41e6aa49510abf1f4b9e', 'request_id': 'req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:49.596475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991 admin admin] The token cache is being invalidate because project daa2a66bfcbd41e6aa49510abf1f4b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:49.596760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:49.601064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f008ef04-e7cb-4cc5-bf22-0eeb4737e991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:49.605815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 941/1862] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:49 2022] DELETE /identity/v3/projects/daa2a66bfcbd41e6aa49510abf1f4b9e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:49.611826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff371781-85ec-4262-a713-2ba97a8b0dfb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:49.612116 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff371781-85ec-4262-a713-2ba97a8b0dfb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:49.612397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff371781-85ec-4262-a713-2ba97a8b0dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:49.632936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ff371781-85ec-4262-a713-2ba97a8b0dfb None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:49.634168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ff371781-85ec-4262-a713-2ba97a8b0dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:49.636100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1e6e55f8-5710-4343-8495-67ac7a2e46be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2258ff812cc848118d33c317c470cf4d', 'request_id': 'req-1e6e55f8-5710-4343-8495-67ac7a2e46be'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:49.652741 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1e6e55f8-5710-4343-8495-67ac7a2e46be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2258ff812cc848118d33c317c470cf4d', 'request_id': 'req-1e6e55f8-5710-4343-8495-67ac7a2e46be'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:49.678987 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1e6e55f8-5710-4343-8495-67ac7a2e46be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2258ff812cc848118d33c317c470cf4d', 'request_id': 'req-1e6e55f8-5710-4343-8495-67ac7a2e46be'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:49.709457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1e6e55f8-5710-4343-8495-67ac7a2e46be admin admin] The token cache is being invalidate because project 2258ff812cc848118d33c317c470cf4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:28:49.709759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1e6e55f8-5710-4343-8495-67ac7a2e46be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:49.710552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1e6e55f8-5710-4343-8495-67ac7a2e46be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:28:49.712683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 922/1863] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:28:49 2022] DELETE /identity/v3/projects/2258ff812cc848118d33c317c470cf4d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:49.717861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38413f-7309-4f7a-ab58-cddc31d15466 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:49.718223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38413f-7309-4f7a-ab58-cddc31d15466 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:49.719662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38413f-7309-4f7a-ab58-cddc31d15466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:49.751959 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-2d38413f-7309-4f7a-ab58-cddc31d15466 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:49.753978 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2d38413f-7309-4f7a-ab58-cddc31d15466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:49.772266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 942/1864] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:49.773548 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24527f-921a-4a90-8873-a6de5e23642a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:49.773798 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24527f-921a-4a90-8873-a6de5e23642a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:49.774213 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24527f-921a-4a90-8873-a6de5e23642a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:49.792145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 923/1865] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:28:49.795128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b4151-62fd-4d1e-84f3-820267043268 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:49.795413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b4151-62fd-4d1e-84f3-820267043268 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:49.795729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b4151-62fd-4d1e-84f3-820267043268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:49.841228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-eb24527f-921a-4a90-8873-a6de5e23642a None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:49.851540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-578b4151-62fd-4d1e-84f3-820267043268 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:49.853097 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-578b4151-62fd-4d1e-84f3-820267043268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:49.853218 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eb24527f-921a-4a90-8873-a6de5e23642a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:49.880452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 943/1866] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:49.881047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 924/1867] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:28:49.881177 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24527f-921a-4a90-8873-a6de5e23642a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:49.882081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b4151-62fd-4d1e-84f3-820267043268 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:49.934208 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b4151-62fd-4d1e-84f3-820267043268 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:49.941581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24527f-921a-4a90-8873-a6de5e23642a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:49.948268 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa18be8-5d58-4e07-81ee-654de7ebe5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:49.949067 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa18be8-5d58-4e07-81ee-654de7ebe5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:49.949357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa18be8-5d58-4e07-81ee-654de7ebe5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:49.949634 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa18be8-5d58-4e07-81ee-654de7ebe5c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:49.950474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa18be8-5d58-4e07-81ee-654de7ebe5c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:28:49.950782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa18be8-5d58-4e07-81ee-654de7ebe5c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:49.961376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d82d9-31a1-4231-8ccf-7f35b89934c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:49.961896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d82d9-31a1-4231-8ccf-7f35b89934c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:49.962195 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d82d9-31a1-4231-8ccf-7f35b89934c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:49.962681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d82d9-31a1-4231-8ccf-7f35b89934c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:49.963450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d82d9-31a1-4231-8ccf-7f35b89934c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:28:49.963798 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d82d9-31a1-4231-8ccf-7f35b89934c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:49.964534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa18be8-5d58-4e07-81ee-654de7ebe5c0 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:49.968388 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d82d9-31a1-4231-8ccf-7f35b89934c0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:49.991271 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 944/1868] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:28:49.993346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 925/1869] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:28:49.998404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d82d9-31a1-4231-8ccf-7f35b89934c0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:49.998595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa18be8-5d58-4e07-81ee-654de7ebe5c0 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:50.010386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa18be8-5d58-4e07-81ee-654de7ebe5c0 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:50.016405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2a49e-020e-4136-be4d-b976ef18b1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:50.017107 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2a49e-020e-4136-be4d-b976ef18b1ec admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.017464 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2a49e-020e-4136-be4d-b976ef18b1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.017937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2a49e-020e-4136-be4d-b976ef18b1ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.018877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2a49e-020e-4136-be4d-b976ef18b1ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:50.020724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2a49e-020e-4136-be4d-b976ef18b1ec admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:50.020952 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d82d9-31a1-4231-8ccf-7f35b89934c0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:50.028324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6c792-8190-4cc6-ba2f-03ea79328339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:50.029142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6c792-8190-4cc6-ba2f-03ea79328339 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.029792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6c792-8190-4cc6-ba2f-03ea79328339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.030218 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6c792-8190-4cc6-ba2f-03ea79328339 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.036056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6c792-8190-4cc6-ba2f-03ea79328339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:50.038800 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6c792-8190-4cc6-ba2f-03ea79328339 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:50.061925 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 926/1870] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:49 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:50.066565 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2a49e-020e-4136-be4d-b976ef18b1ec admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:50.082481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 945/1871] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:49 2022] POST /identity/v3/projects => generated 383 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:50.086585 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6c792-8190-4cc6-ba2f-03ea79328339 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:50.145126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2a49e-020e-4136-be4d-b976ef18b1ec admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:50.197274 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6c792-8190-4cc6-ba2f-03ea79328339 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:50.201645 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:50.206895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af94ca-6338-4aa6-bbf4-1877c8ffe9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:50.207634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af94ca-6338-4aa6-bbf4-1877c8ffe9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.207922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af94ca-6338-4aa6-bbf4-1877c8ffe9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.208194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af94ca-6338-4aa6-bbf4-1877c8ffe9b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.208828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af94ca-6338-4aa6-bbf4-1877c8ffe9b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:50.210742 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af94ca-6338-4aa6-bbf4-1877c8ffe9b4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:50.215358 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c0186-758a-4919-902d-338836e3437a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:50.216166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c0186-758a-4919-902d-338836e3437a admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.216485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c0186-758a-4919-902d-338836e3437a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.216779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c0186-758a-4919-902d-338836e3437a admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.217746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c0186-758a-4919-902d-338836e3437a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:50.219796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c0186-758a-4919-902d-338836e3437a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:50.223146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:50.239815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 946/1872] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:50 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:50.242898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af94ca-6338-4aa6-bbf4-1877c8ffe9b4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:50.247858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 927/1873] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:50 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:50.248744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c0186-758a-4919-902d-338836e3437a admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:50.253376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af94ca-6338-4aa6-bbf4-1877c8ffe9b4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:50.259070 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c0186-758a-4919-902d-338836e3437a admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:50.259949 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e075a7-9394-4197-ad89-3ca4e8115d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:50.260559 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e075a7-9394-4197-ad89-3ca4e8115d2a admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.260833 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e075a7-9394-4197-ad89-3ca4e8115d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.261098 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e075a7-9394-4197-ad89-3ca4e8115d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.261744 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e075a7-9394-4197-ad89-3ca4e8115d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:50.263389 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e075a7-9394-4197-ad89-3ca4e8115d2a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:50.264588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89e830-cec5-4f06-93ab-cfcbbe75b392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:50.265365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89e830-cec5-4f06-93ab-cfcbbe75b392 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.266546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89e830-cec5-4f06-93ab-cfcbbe75b392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.266831 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89e830-cec5-4f06-93ab-cfcbbe75b392 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.267474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89e830-cec5-4f06-93ab-cfcbbe75b392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:50.269116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89e830-cec5-4f06-93ab-cfcbbe75b392 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:50.275621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 947/1874] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:50.279473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e075a7-9394-4197-ad89-3ca4e8115d2a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:50.284346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 928/1875] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:50.288172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89e830-cec5-4f06-93ab-cfcbbe75b392 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:50.291302 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e075a7-9394-4197-ad89-3ca4e8115d2a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:50.296893 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94b509-bf41-4236-a96c-b7d8a067dbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:50.297520 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94b509-bf41-4236-a96c-b7d8a067dbbc admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.297799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94b509-bf41-4236-a96c-b7d8a067dbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.298070 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94b509-bf41-4236-a96c-b7d8a067dbbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.298188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89e830-cec5-4f06-93ab-cfcbbe75b392 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:50.298708 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94b509-bf41-4236-a96c-b7d8a067dbbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:50.300352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94b509-bf41-4236-a96c-b7d8a067dbbc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:50.305283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0217e4-e4be-409c-812e-7d03cc3ec9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:50.306049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0217e4-e4be-409c-812e-7d03cc3ec9ec admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.306343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0217e4-e4be-409c-812e-7d03cc3ec9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.306623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0217e4-e4be-409c-812e-7d03cc3ec9ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.307268 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0217e4-e4be-409c-812e-7d03cc3ec9ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:50.308939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0217e4-e4be-409c-812e-7d03cc3ec9ec admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:50.311428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 948/1876] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:28:50.315756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94b509-bf41-4236-a96c-b7d8a067dbbc admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:50.325487 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 929/1877] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:28:50.326204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94b509-bf41-4236-a96c-b7d8a067dbbc admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:50.329966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0217e4-e4be-409c-812e-7d03cc3ec9ec admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:50.332091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84554a6a-a3a9-4165-8ebf-e40e694724e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:50.332910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84554a6a-a3a9-4165-8ebf-e40e694724e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.333202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84554a6a-a3a9-4165-8ebf-e40e694724e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.333575 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84554a6a-a3a9-4165-8ebf-e40e694724e3 admin admin] PATH_INFO: `/v3/projects/a510edc56c6d423097a3ca7d1d992402/users/b2209065311242f2a6fc6326bcb49682/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.339681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0217e4-e4be-409c-812e-7d03cc3ec9ec admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:50.344538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561f75c-4d42-41ff-bf52-8cfbdca2c676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:50.345248 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561f75c-4d42-41ff-bf52-8cfbdca2c676 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.345535 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561f75c-4d42-41ff-bf52-8cfbdca2c676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.345811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561f75c-4d42-41ff-bf52-8cfbdca2c676 admin admin] PATH_INFO: `/v3/projects/96222758254a4bd6971a94e0a7d058a4/users/604bae081ccf44e5ba2ffab401dcd738/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.350158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84554a6a-a3a9-4165-8ebf-e40e694724e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a510edc56c6d423097a3ca7d1d992402, user_id=b2209065311242f2a6fc6326bcb49682, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:50.353017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84554a6a-a3a9-4165-8ebf-e40e694724e3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:50.362505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561f75c-4d42-41ff-bf52-8cfbdca2c676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96222758254a4bd6971a94e0a7d058a4, user_id=604bae081ccf44e5ba2ffab401dcd738, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:50.365221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561f75c-4d42-41ff-bf52-8cfbdca2c676 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:50.368309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 949/1878] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:50 2022] PUT /identity/v3/projects/a510edc56c6d423097a3ca7d1d992402/users/b2209065311242f2a6fc6326bcb49682/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:28:50.380666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744f73d-c6fa-42d7-a524-0c9053d2c9ca None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.381165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744f73d-c6fa-42d7-a524-0c9053d2c9ca None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.381663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744f73d-c6fa-42d7-a524-0c9053d2c9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.400476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 930/1879] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:50 2022] PUT /identity/v3/projects/96222758254a4bd6971a94e0a7d058a4/users/604bae081ccf44e5ba2ffab401dcd738/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:28:50.411089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca37d0-db12-4e4b-b158-c978cbb6940c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.411471 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca37d0-db12-4e4b-b158-c978cbb6940c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.412103 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca37d0-db12-4e4b-b158-c978cbb6940c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.449174 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b2ca37d0-db12-4e4b-b158-c978cbb6940c None None] MFA Rules not processed for user `604bae081ccf44e5ba2ffab401dcd738`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:50.450576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b2ca37d0-db12-4e4b-b158-c978cbb6940c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:50.463821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0744f73d-c6fa-42d7-a524-0c9053d2c9ca None None] MFA Rules not processed for user `b2209065311242f2a6fc6326bcb49682`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:50.463821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0744f73d-c6fa-42d7-a524-0c9053d2c9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:50.621558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 950/1880] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:28:50.626234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744f73d-c6fa-42d7-a524-0c9053d2c9ca None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:50.632988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 931/1881] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:28:50.681993 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca37d0-db12-4e4b-b158-c978cbb6940c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:50.686588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b2ca37d0-db12-4e4b-b158-c978cbb6940c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:50.811869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744f73d-c6fa-42d7-a524-0c9053d2c9ca None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:50.819210 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377f93a-4679-4a71-aaee-93baf6ddf91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:50.822075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377f93a-4679-4a71-aaee-93baf6ddf91f admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.822361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377f93a-4679-4a71-aaee-93baf6ddf91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.822634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377f93a-4679-4a71-aaee-93baf6ddf91f admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.823279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377f93a-4679-4a71-aaee-93baf6ddf91f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:50.825915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377f93a-4679-4a71-aaee-93baf6ddf91f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:50.831606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca37d0-db12-4e4b-b158-c978cbb6940c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:50.837631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc2f42-80d5-4eb4-a9d3-7a80a395c19d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61407406ef242a78030f79532fe2247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:50.838376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc2f42-80d5-4eb4-a9d3-7a80a395c19d service glance] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.838660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc2f42-80d5-4eb4-a9d3-7a80a395c19d service glance] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.838979 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc2f42-80d5-4eb4-a9d3-7a80a395c19d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.849915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc2f42-80d5-4eb4-a9d3-7a80a395c19d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:50.852243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc2f42-80d5-4eb4-a9d3-7a80a395c19d service glance] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:50.863500 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 932/1882] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:28:50.876488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 951/1883] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:28:50 2022] POST /identity/v3/projects => generated 385 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:28:50.877217 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc2f42-80d5-4eb4-a9d3-7a80a395c19d service glance] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:50.899928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc2f42-80d5-4eb4-a9d3-7a80a395c19d service glance] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:50.903962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:50.965157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf0cbd-01cf-4746-8800-2beb2a34ac79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:50.965936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf0cbd-01cf-4746-8800-2beb2a34ac79 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:50.965936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf0cbd-01cf-4746-8800-2beb2a34ac79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:50.966340 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf0cbd-01cf-4746-8800-2beb2a34ac79 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:50.978287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf0cbd-01cf-4746-8800-2beb2a34ac79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:50.982823 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf0cbd-01cf-4746-8800-2beb2a34ac79 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:51.049397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377f93a-4679-4a71-aaee-93baf6ddf91f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:51.058755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 933/1884] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:28:50 2022] POST /identity/v3/users => generated 299 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:28:51.063979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377f93a-4679-4a71-aaee-93baf6ddf91f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:51.071449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f152b-029f-4edb-aa0b-0f9985450376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:51.072398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f152b-029f-4edb-aa0b-0f9985450376 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:51.072814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f152b-029f-4edb-aa0b-0f9985450376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:51.073261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f152b-029f-4edb-aa0b-0f9985450376 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:51.074307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f152b-029f-4edb-aa0b-0f9985450376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:51.076819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f152b-029f-4edb-aa0b-0f9985450376 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:51.096358 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 952/1885] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:51.100333 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf0cbd-01cf-4746-8800-2beb2a34ac79 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:51.118485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf0cbd-01cf-4746-8800-2beb2a34ac79 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:51.141337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8fc49-1c77-49f8-afe8-a386348d20b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:51.142369 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8fc49-1c77-49f8-afe8-a386348d20b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:51.142861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8fc49-1c77-49f8-afe8-a386348d20b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:51.143442 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8fc49-1c77-49f8-afe8-a386348d20b8 admin admin] PATH_INFO: `/v3/projects/1d4152054c684b2ab1acb5b03846bf62/users/975bdefb199e424f90761df06b68aecf/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:51.170230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8fc49-1c77-49f8-afe8-a386348d20b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4152054c684b2ab1acb5b03846bf62, user_id=975bdefb199e424f90761df06b68aecf, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:51.174650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8fc49-1c77-49f8-afe8-a386348d20b8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:51.211357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 934/1886] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:51 2022] PUT /identity/v3/projects/1d4152054c684b2ab1acb5b03846bf62/users/975bdefb199e424f90761df06b68aecf/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:51.215821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f152b-029f-4edb-aa0b-0f9985450376 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:51.233710 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f152b-029f-4edb-aa0b-0f9985450376 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:51.254165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:28:51.324222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfeac1-4d2c-493e-992a-ed24cf2921f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:51.324799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfeac1-4d2c-493e-992a-ed24cf2921f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:51.325084 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfeac1-4d2c-493e-992a-ed24cf2921f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:51.325364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfeac1-4d2c-493e-992a-ed24cf2921f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:51.326695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfeac1-4d2c-493e-992a-ed24cf2921f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:51.329019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfeac1-4d2c-493e-992a-ed24cf2921f4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:51.347938 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 953/1887] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:28:51.351431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8fc49-1c77-49f8-afe8-a386348d20b8 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:51.363958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8fc49-1c77-49f8-afe8-a386348d20b8 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:51.376843 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0becb-aae2-47c0-b411-3f5de336e6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:51.378820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0becb-aae2-47c0-b411-3f5de336e6c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:51.379214 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0becb-aae2-47c0-b411-3f5de336e6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:51.379672 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0becb-aae2-47c0-b411-3f5de336e6c3 admin admin] PATH_INFO: `/v3/projects/1d4152054c684b2ab1acb5b03846bf62/users/975bdefb199e424f90761df06b68aecf/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:51.382139 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0becb-aae2-47c0-b411-3f5de336e6c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4152054c684b2ab1acb5b03846bf62, user_id=975bdefb199e424f90761df06b68aecf, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:51.392243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0becb-aae2-47c0-b411-3f5de336e6c3 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:51.423618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 935/1888] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:28:51 2022] PUT /identity/v3/projects/1d4152054c684b2ab1acb5b03846bf62/users/975bdefb199e424f90761df06b68aecf/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:28:51.425711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04447d-5dc5-487a-9e77-f143f7f95fea None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:51.426349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04447d-5dc5-487a-9e77-f143f7f95fea None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:51.427008 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04447d-5dc5-487a-9e77-f143f7f95fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:51.466685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-de04447d-5dc5-487a-9e77-f143f7f95fea None None] MFA Rules not processed for user `975bdefb199e424f90761df06b68aecf`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:28:51.469335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-de04447d-5dc5-487a-9e77-f143f7f95fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:51.665850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 954/1889] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:28:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:28:51.689901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0becb-aae2-47c0-b411-3f5de336e6c3 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:28:51.691812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-dee0becb-aae2-47c0-b411-3f5de336e6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:28:51.854070 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0becb-aae2-47c0-b411-3f5de336e6c3 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:28:51.868349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b580d3f-dd55-44a3-891a-0e5d419e5cc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:28:51.869363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b580d3f-dd55-44a3-891a-0e5d419e5cc0 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:28:51.869659 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b580d3f-dd55-44a3-891a-0e5d419e5cc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:28:51.869977 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b580d3f-dd55-44a3-891a-0e5d419e5cc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:28:51.991976 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b580d3f-dd55-44a3-891a-0e5d419e5cc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:28:52.000662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b580d3f-dd55-44a3-891a-0e5d419e5cc0 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:28:52.027340 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 936/1890] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:28:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:01.274105 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04447d-5dc5-487a-9e77-f143f7f95fea None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:01.290194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04447d-5dc5-487a-9e77-f143f7f95fea None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:01.297661 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d271980-0d50-4ad7-b21d-5c65780617eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:01.298361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d271980-0d50-4ad7-b21d-5c65780617eb service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:01.311602 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d271980-0d50-4ad7-b21d-5c65780617eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:01.311602 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d271980-0d50-4ad7-b21d-5c65780617eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:01.341910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d271980-0d50-4ad7-b21d-5c65780617eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:01.349504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d271980-0d50-4ad7-b21d-5c65780617eb service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:01.381907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 955/1891] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:02.020642 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:INSERT=5 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:08.653362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b580d3f-dd55-44a3-891a-0e5d419e5cc0 service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:08.721437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b580d3f-dd55-44a3-891a-0e5d419e5cc0 service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:08.728014 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f992ded-f8a8-44af-be66-39ecd3fd8c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:08.729544 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f992ded-f8a8-44af-be66-39ecd3fd8c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:08.730096 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f992ded-f8a8-44af-be66-39ecd3fd8c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:08.730636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f992ded-f8a8-44af-be66-39ecd3fd8c5e admin admin] PATH_INFO: `/v3/users/b2209065311242f2a6fc6326bcb49682` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:08.732373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f992ded-f8a8-44af-be66-39ecd3fd8c5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2209065311242f2a6fc6326bcb49682)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:08.741054 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f992ded-f8a8-44af-be66-39ecd3fd8c5e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:08.812375 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-8f992ded-f8a8-44af-be66-39ecd3fd8c5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2209065311242f2a6fc6326bcb49682'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:08.825109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-8f992ded-f8a8-44af-be66-39ecd3fd8c5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2209065311242f2a6fc6326bcb49682'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:08.834035 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-8f992ded-f8a8-44af-be66-39ecd3fd8c5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2209065311242f2a6fc6326bcb49682'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:08.834711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-8f992ded-f8a8-44af-be66-39ecd3fd8c5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2209065311242f2a6fc6326bcb49682'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:08.850157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-8f992ded-f8a8-44af-be66-39ecd3fd8c5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2209065311242f2a6fc6326bcb49682'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:08.862338 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 937/1892] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:08 2022] DELETE /identity/v3/users/b2209065311242f2a6fc6326bcb49682 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:08.866877 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf55403-3f1b-479b-9cf8-caeab7545494 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:08.867377 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf55403-3f1b-479b-9cf8-caeab7545494 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:08.867492 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf55403-3f1b-479b-9cf8-caeab7545494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:08.886542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8cf55403-3f1b-479b-9cf8-caeab7545494 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:08.887690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8cf55403-3f1b-479b-9cf8-caeab7545494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:08.950192 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 956/1893] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:08.960869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f992ded-f8a8-44af-be66-39ecd3fd8c5e admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:08.962324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8f992ded-f8a8-44af-be66-39ecd3fd8c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:09.028045 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f992ded-f8a8-44af-be66-39ecd3fd8c5e admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:09.032165 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade0afe-813f-4009-855a-68dccd350386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:09.032771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade0afe-813f-4009-855a-68dccd350386 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:09.033048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade0afe-813f-4009-855a-68dccd350386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:09.033318 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade0afe-813f-4009-855a-68dccd350386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:09.042576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade0afe-813f-4009-855a-68dccd350386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:09.044043 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade0afe-813f-4009-855a-68dccd350386 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:09.054027 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 938/1894] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:09.670831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf55403-3f1b-479b-9cf8-caeab7545494 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:09.673204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8cf55403-3f1b-479b-9cf8-caeab7545494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:09.696282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf55403-3f1b-479b-9cf8-caeab7545494 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:09.708486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:09.711001 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:09.711400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:09.712964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] PATH_INFO: `/v3/projects/a510edc56c6d423097a3ca7d1d992402` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:09.718080 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a510edc56c6d423097a3ca7d1d992402)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:09.724600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:09.759409 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:09.814165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a510edc56c6d423097a3ca7d1d992402', 'request_id': 'req-b129fc81-51d6-4296-bb10-55fb708d894f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:09.837093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a510edc56c6d423097a3ca7d1d992402', 'request_id': 'req-b129fc81-51d6-4296-bb10-55fb708d894f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:09.849651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a510edc56c6d423097a3ca7d1d992402', 'request_id': 'req-b129fc81-51d6-4296-bb10-55fb708d894f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:09.865897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] The token cache is being invalidate because project a510edc56c6d423097a3ca7d1d992402 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:09.866446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:09.867681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:09.873447 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 957/1895] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:09 2022] DELETE /identity/v3/projects/a510edc56c6d423097a3ca7d1d992402 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:09.873628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade0afe-813f-4009-855a-68dccd350386 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:09.884878 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade0afe-813f-4009-855a-68dccd350386 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:09.912298 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:09.979925 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b73e51-0c76-4350-a83c-9711fb60cfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:09.980885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b73e51-0c76-4350-a83c-9711fb60cfab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:09.981359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b73e51-0c76-4350-a83c-9711fb60cfab admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:09.981949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b73e51-0c76-4350-a83c-9711fb60cfab admin admin] PATH_INFO: `/v3/users/975bdefb199e424f90761df06b68aecf` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:09.983913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b73e51-0c76-4350-a83c-9711fb60cfab admin admin] RBAC: Authorizing `identity:delete_user(user_id=975bdefb199e424f90761df06b68aecf)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:09.986431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b73e51-0c76-4350-a83c-9711fb60cfab admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:10.051523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-47b73e51-0c76-4350-a83c-9711fb60cfab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '975bdefb199e424f90761df06b68aecf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:10.064068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-47b73e51-0c76-4350-a83c-9711fb60cfab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '975bdefb199e424f90761df06b68aecf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:10.071448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-47b73e51-0c76-4350-a83c-9711fb60cfab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '975bdefb199e424f90761df06b68aecf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:10.072203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-47b73e51-0c76-4350-a83c-9711fb60cfab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '975bdefb199e424f90761df06b68aecf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:10.079359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-47b73e51-0c76-4350-a83c-9711fb60cfab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '975bdefb199e424f90761df06b68aecf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:10.085235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 939/1896] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:09 2022] DELETE /identity/v3/users/975bdefb199e424f90761df06b68aecf => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:10.366111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:10.367823 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:10.386535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129fc81-51d6-4296-bb10-55fb708d894f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:10.469039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:10.470286 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:10.470572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:10.470846 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6 admin admin] PATH_INFO: `/v3/projects/1d4152054c684b2ab1acb5b03846bf62` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:10.472023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d4152054c684b2ab1acb5b03846bf62)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:10.474398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:10.543713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d4152054c684b2ab1acb5b03846bf62', 'request_id': 'req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:10.550658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d4152054c684b2ab1acb5b03846bf62', 'request_id': 'req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:10.558304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d4152054c684b2ab1acb5b03846bf62', 'request_id': 'req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:10.570359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6 admin admin] The token cache is being invalidate because project 1d4152054c684b2ab1acb5b03846bf62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:10.570454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:10.571115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5dd3b0cb-cd70-4338-98ea-dc675054f6d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:10.572648 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 958/1897] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:10 2022] DELETE /identity/v3/projects/1d4152054c684b2ab1acb5b03846bf62 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:10.579749 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72786ee-9c45-4eee-9f9a-746661c88178 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:10.580102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72786ee-9c45-4eee-9f9a-746661c88178 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:10.580498 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72786ee-9c45-4eee-9f9a-746661c88178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:10.600371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c72786ee-9c45-4eee-9f9a-746661c88178 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:10.601573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c72786ee-9c45-4eee-9f9a-746661c88178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:10.666686 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 940/1898] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:10.675416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f681d-cc2a-4fc6-9269-3e7c96fa866d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:10.675740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f681d-cc2a-4fc6-9269-3e7c96fa866d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:10.676019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f681d-cc2a-4fc6-9269-3e7c96fa866d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:10.696880 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2c9f681d-cc2a-4fc6-9269-3e7c96fa866d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:10.698029 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2c9f681d-cc2a-4fc6-9269-3e7c96fa866d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:10.706665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 959/1899] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:10.710812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72786ee-9c45-4eee-9f9a-746661c88178 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:10.721501 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72786ee-9c45-4eee-9f9a-746661c88178 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:10.726091 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946487c1-da12-43cc-b090-82b22cf70ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:10.726687 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946487c1-da12-43cc-b090-82b22cf70ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:10.726963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946487c1-da12-43cc-b090-82b22cf70ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:10.727233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946487c1-da12-43cc-b090-82b22cf70ccf admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:10.728009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946487c1-da12-43cc-b090-82b22cf70ccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:29:10.728318 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946487c1-da12-43cc-b090-82b22cf70ccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:10.729926 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946487c1-da12-43cc-b090-82b22cf70ccf admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:10.743105 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 941/1900] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:29:10.747440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f681d-cc2a-4fc6-9269-3e7c96fa866d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:10.757329 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f681d-cc2a-4fc6-9269-3e7c96fa866d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:10.762347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7fe7d-275e-48a8-a904-ed0ea522f5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:10.762956 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7fe7d-275e-48a8-a904-ed0ea522f5fd admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:10.763239 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7fe7d-275e-48a8-a904-ed0ea522f5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:10.763508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7fe7d-275e-48a8-a904-ed0ea522f5fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:10.764222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7fe7d-275e-48a8-a904-ed0ea522f5fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:10.766707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7fe7d-275e-48a8-a904-ed0ea522f5fd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:10.821087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 960/1901] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:10 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:29:10.822320 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946487c1-da12-43cc-b090-82b22cf70ccf admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:10.834467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946487c1-da12-43cc-b090-82b22cf70ccf admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:10.871163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:10.919518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0991b5f-2067-4a8d-9b4f-9297d0e40d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:10.920890 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0991b5f-2067-4a8d-9b4f-9297d0e40d7f admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:10.921324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0991b5f-2067-4a8d-9b4f-9297d0e40d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:10.921747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0991b5f-2067-4a8d-9b4f-9297d0e40d7f admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:10.922541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0991b5f-2067-4a8d-9b4f-9297d0e40d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:10.925068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0991b5f-2067-4a8d-9b4f-9297d0e40d7f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:10.973853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 942/1902] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:10 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:10.977815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7fe7d-275e-48a8-a904-ed0ea522f5fd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:10.992223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7fe7d-275e-48a8-a904-ed0ea522f5fd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:11.001083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ada77-5521-4acd-a1af-a8e62c2a11e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:11.001964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ada77-5521-4acd-a1af-a8e62c2a11e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:11.002403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ada77-5521-4acd-a1af-a8e62c2a11e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:11.002872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ada77-5521-4acd-a1af-a8e62c2a11e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:11.017363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ada77-5521-4acd-a1af-a8e62c2a11e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:11.019288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ada77-5521-4acd-a1af-a8e62c2a11e1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:11.026568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:11.049680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 961/1903] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:11.053789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0991b5f-2067-4a8d-9b4f-9297d0e40d7f admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:11.071069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0991b5f-2067-4a8d-9b4f-9297d0e40d7f admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:11.078754 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f3e5a-6a1e-4814-9f3f-7d5e4aee1f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:11.079732 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f3e5a-6a1e-4814-9f3f-7d5e4aee1f22 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:11.080189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f3e5a-6a1e-4814-9f3f-7d5e4aee1f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:11.080627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f3e5a-6a1e-4814-9f3f-7d5e4aee1f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:11.081427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f3e5a-6a1e-4814-9f3f-7d5e4aee1f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:11.083273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f3e5a-6a1e-4814-9f3f-7d5e4aee1f22 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:11.098254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 943/1904] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:11.102643 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ada77-5521-4acd-a1af-a8e62c2a11e1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:11.120402 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ada77-5521-4acd-a1af-a8e62c2a11e1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:11.129038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f06bea-6e69-4837-910d-9c428a65e0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:11.129719 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f06bea-6e69-4837-910d-9c428a65e0fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:11.130000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f06bea-6e69-4837-910d-9c428a65e0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:11.130276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f06bea-6e69-4837-910d-9c428a65e0fd admin admin] PATH_INFO: `/v3/projects/08fff01b75b04446bcf402130343bc77/users/2d2fa12dc34d4ceeaa748e0e92aa9791/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:11.147034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f06bea-6e69-4837-910d-9c428a65e0fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=08fff01b75b04446bcf402130343bc77, user_id=2d2fa12dc34d4ceeaa748e0e92aa9791, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:11.149687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f06bea-6e69-4837-910d-9c428a65e0fd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:11.165521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 962/1905] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:11 2022] PUT /identity/v3/projects/08fff01b75b04446bcf402130343bc77/users/2d2fa12dc34d4ceeaa748e0e92aa9791/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:11.178118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee564c-3c35-4612-906e-d0768a32bcd1 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:11.178466 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee564c-3c35-4612-906e-d0768a32bcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:11.178817 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee564c-3c35-4612-906e-d0768a32bcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:11.211890 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-48ee564c-3c35-4612-906e-d0768a32bcd1 None None] MFA Rules not processed for user `2d2fa12dc34d4ceeaa748e0e92aa9791`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:11.213107 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-48ee564c-3c35-4612-906e-d0768a32bcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:11.279467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 944/1906] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:11.284852 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f06bea-6e69-4837-910d-9c428a65e0fd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:11.345114 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f06bea-6e69-4837-910d-9c428a65e0fd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:11.349690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f104af-b7a3-46a6-92b7-f2819bf34c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:11.350288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f104af-b7a3-46a6-92b7-f2819bf34c3a admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:11.350566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f104af-b7a3-46a6-92b7-f2819bf34c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:11.350836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f104af-b7a3-46a6-92b7-f2819bf34c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:11.351453 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f104af-b7a3-46a6-92b7-f2819bf34c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:11.353381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f104af-b7a3-46a6-92b7-f2819bf34c3a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:11.384580 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 963/1907] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:11 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:11.386321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee564c-3c35-4612-906e-d0768a32bcd1 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:11.398758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee564c-3c35-4612-906e-d0768a32bcd1 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:11.472053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6fff1-99c0-4e08-bfce-bbe0f3e9d279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:11.472802 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6fff1-99c0-4e08-bfce-bbe0f3e9d279 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:11.473179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6fff1-99c0-4e08-bfce-bbe0f3e9d279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:11.473623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6fff1-99c0-4e08-bfce-bbe0f3e9d279 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:11.474471 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6fff1-99c0-4e08-bfce-bbe0f3e9d279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:11.476971 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6fff1-99c0-4e08-bfce-bbe0f3e9d279 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:11.515314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 945/1908] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:11 2022] POST /identity/v3/users => generated 294 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:29:11.517931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f104af-b7a3-46a6-92b7-f2819bf34c3a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:11.528259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f104af-b7a3-46a6-92b7-f2819bf34c3a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:11.540095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880ce42-1a93-4573-9d20-e18b194dab1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:11.540718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880ce42-1a93-4573-9d20-e18b194dab1e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:11.540999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880ce42-1a93-4573-9d20-e18b194dab1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:11.541678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880ce42-1a93-4573-9d20-e18b194dab1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:11.541923 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f880ce42-1a93-4573-9d20-e18b194dab1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:11.543523 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f880ce42-1a93-4573-9d20-e18b194dab1e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:11.557288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 964/1909] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:11.561459 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6fff1-99c0-4e08-bfce-bbe0f3e9d279 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:11.573408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6fff1-99c0-4e08-bfce-bbe0f3e9d279 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:11.579399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70d0a9-86eb-4c84-acf6-9d568bbba50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:11.580191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d0a9-86eb-4c84-acf6-9d568bbba50f admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:11.580576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d0a9-86eb-4c84-acf6-9d568bbba50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:11.580950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d0a9-86eb-4c84-acf6-9d568bbba50f admin admin] PATH_INFO: `/v3/projects/8d1fd16d8bf04f90bb348a26273d3f5a/users/7e2700131d204ee393a5602068a8bb4a/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:11.606068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70d0a9-86eb-4c84-acf6-9d568bbba50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1fd16d8bf04f90bb348a26273d3f5a, user_id=7e2700131d204ee393a5602068a8bb4a, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:11.609513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70d0a9-86eb-4c84-acf6-9d568bbba50f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:11.625375 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 946/1910] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:11 2022] PUT /identity/v3/projects/8d1fd16d8bf04f90bb348a26273d3f5a/users/7e2700131d204ee393a5602068a8bb4a/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:11.629443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880ce42-1a93-4573-9d20-e18b194dab1e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:11.646539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880ce42-1a93-4573-9d20-e18b194dab1e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:11.713040 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929af39f-4a53-408c-8abe-e42977cb0894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:11.713843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929af39f-4a53-408c-8abe-e42977cb0894 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:11.714337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929af39f-4a53-408c-8abe-e42977cb0894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:11.714722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929af39f-4a53-408c-8abe-e42977cb0894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:11.715562 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929af39f-4a53-408c-8abe-e42977cb0894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:11.717500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929af39f-4a53-408c-8abe-e42977cb0894 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:11.729765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 965/1911] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:11.733674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70d0a9-86eb-4c84-acf6-9d568bbba50f admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:11.739208 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:11.745619 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70d0a9-86eb-4c84-acf6-9d568bbba50f admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:11.751087 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee3b41-46b1-47ed-a94f-e703edefe216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:11.751769 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee3b41-46b1-47ed-a94f-e703edefe216 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:11.752172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee3b41-46b1-47ed-a94f-e703edefe216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:11.752451 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee3b41-46b1-47ed-a94f-e703edefe216 admin admin] PATH_INFO: `/v3/projects/8d1fd16d8bf04f90bb348a26273d3f5a/users/7e2700131d204ee393a5602068a8bb4a/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:11.754340 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee3b41-46b1-47ed-a94f-e703edefe216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1fd16d8bf04f90bb348a26273d3f5a, user_id=7e2700131d204ee393a5602068a8bb4a, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:11.757063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee3b41-46b1-47ed-a94f-e703edefe216 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:11.768463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 947/1912] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:11 2022] PUT /identity/v3/projects/8d1fd16d8bf04f90bb348a26273d3f5a/users/7e2700131d204ee393a5602068a8bb4a/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:11.778874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690148c1-fec7-4d94-bdc3-d7667753bae0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:11.779204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690148c1-fec7-4d94-bdc3-d7667753bae0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:11.779439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690148c1-fec7-4d94-bdc3-d7667753bae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:11.808236 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-690148c1-fec7-4d94-bdc3-d7667753bae0 None None] MFA Rules not processed for user `7e2700131d204ee393a5602068a8bb4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:11.809457 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-690148c1-fec7-4d94-bdc3-d7667753bae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:11.841038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:11.881561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 966/1913] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:11.893567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee3b41-46b1-47ed-a94f-e703edefe216 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:11.895227 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-72ee3b41-46b1-47ed-a94f-e703edefe216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:11.951477 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee3b41-46b1-47ed-a94f-e703edefe216 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:11.955340 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ecae0-9ee0-4ff6-953e-2d4fd68a3a5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:11.956034 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ecae0-9ee0-4ff6-953e-2d4fd68a3a5e service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:11.956315 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ecae0-9ee0-4ff6-953e-2d4fd68a3a5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:11.956591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ecae0-9ee0-4ff6-953e-2d4fd68a3a5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:12.027146 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ecae0-9ee0-4ff6-953e-2d4fd68a3a5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:12.028636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ecae0-9ee0-4ff6-953e-2d4fd68a3a5e service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:12.038616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 948/1914] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:12.696694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690148c1-fec7-4d94-bdc3-d7667753bae0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:12.699717 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-690148c1-fec7-4d94-bdc3-d7667753bae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:12.776866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690148c1-fec7-4d94-bdc3-d7667753bae0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:12.782724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7cd0f-69a7-449d-8f59-023dc03f9ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:12.783368 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7cd0f-69a7-449d-8f59-023dc03f9ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:12.783689 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7cd0f-69a7-449d-8f59-023dc03f9ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:12.783991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7cd0f-69a7-449d-8f59-023dc03f9ead admin admin] PATH_INFO: `/v3/users/2179bb8a78874947ac676c2f7f9329fd` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:12.785121 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7cd0f-69a7-449d-8f59-023dc03f9ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=2179bb8a78874947ac676c2f7f9329fd)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:12.787033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7cd0f-69a7-449d-8f59-023dc03f9ead admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:12.833203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-07c7cd0f-69a7-449d-8f59-023dc03f9ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2179bb8a78874947ac676c2f7f9329fd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:12.839828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-07c7cd0f-69a7-449d-8f59-023dc03f9ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2179bb8a78874947ac676c2f7f9329fd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:12.851951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-07c7cd0f-69a7-449d-8f59-023dc03f9ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2179bb8a78874947ac676c2f7f9329fd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:12.859477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-07c7cd0f-69a7-449d-8f59-023dc03f9ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2179bb8a78874947ac676c2f7f9329fd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:12.860160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-07c7cd0f-69a7-449d-8f59-023dc03f9ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2179bb8a78874947ac676c2f7f9329fd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:12.865857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 967/1915] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:12 2022] DELETE /identity/v3/users/2179bb8a78874947ac676c2f7f9329fd => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:12.870492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506c1c4-44c2-4d3c-97a7-a575a8c7dd3f None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:12.870795 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506c1c4-44c2-4d3c-97a7-a575a8c7dd3f None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:12.871098 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506c1c4-44c2-4d3c-97a7-a575a8c7dd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:12.890598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-3506c1c4-44c2-4d3c-97a7-a575a8c7dd3f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:12.891849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3506c1c4-44c2-4d3c-97a7-a575a8c7dd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:12.953517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 949/1916] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:12.963194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7cd0f-69a7-449d-8f59-023dc03f9ead admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:12.964626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-07c7cd0f-69a7-449d-8f59-023dc03f9ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:13.017261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:13.023538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7cd0f-69a7-449d-8f59-023dc03f9ead admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:13.027990 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61362236-ddc5-40b5-bafc-b46b823e8ff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:13.028593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61362236-ddc5-40b5-bafc-b46b823e8ff7 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:13.028875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61362236-ddc5-40b5-bafc-b46b823e8ff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:13.029150 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61362236-ddc5-40b5-bafc-b46b823e8ff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:13.038733 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61362236-ddc5-40b5-bafc-b46b823e8ff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:13.040214 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61362236-ddc5-40b5-bafc-b46b823e8ff7 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:13.050300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 968/1917] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:13.336550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506c1c4-44c2-4d3c-97a7-a575a8c7dd3f None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:13.338332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3506c1c4-44c2-4d3c-97a7-a575a8c7dd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:13.351093 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506c1c4-44c2-4d3c-97a7-a575a8c7dd3f None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:13.355804 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:13.356442 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:13.356723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:13.356997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] PATH_INFO: `/v3/projects/148b03d0d93f428ca93a67e409078a9c` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:13.358285 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=148b03d0d93f428ca93a67e409078a9c)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:13.360364 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:13.402000 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:13.410970 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '148b03d0d93f428ca93a67e409078a9c', 'request_id': 'req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:13.418318 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '148b03d0d93f428ca93a67e409078a9c', 'request_id': 'req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:13.426147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '148b03d0d93f428ca93a67e409078a9c', 'request_id': 'req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:13.438696 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] The token cache is being invalidate because project 148b03d0d93f428ca93a67e409078a9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:13.439067 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:13.439821 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:13.441713 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 950/1918] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:13 2022] DELETE /identity/v3/projects/148b03d0d93f428ca93a67e409078a9c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:13.445631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61362236-ddc5-40b5-bafc-b46b823e8ff7 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:13.463715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61362236-ddc5-40b5-bafc-b46b823e8ff7 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:13.534685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8ca20-983d-4641-80f6-97c8a7672187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:13.535186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8ca20-983d-4641-80f6-97c8a7672187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:13.535471 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8ca20-983d-4641-80f6-97c8a7672187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:13.535763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8ca20-983d-4641-80f6-97c8a7672187 admin admin] PATH_INFO: `/v3/users/0c6f00d24905449c887b1a2f8149b4ff` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:13.536898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8ca20-983d-4641-80f6-97c8a7672187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c6f00d24905449c887b1a2f8149b4ff)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:13.538973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8ca20-983d-4641-80f6-97c8a7672187 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:13.601105 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8da8ca20-983d-4641-80f6-97c8a7672187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c6f00d24905449c887b1a2f8149b4ff'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:13.609275 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8da8ca20-983d-4641-80f6-97c8a7672187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c6f00d24905449c887b1a2f8149b4ff'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:13.628632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8da8ca20-983d-4641-80f6-97c8a7672187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c6f00d24905449c887b1a2f8149b4ff'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:13.637676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8da8ca20-983d-4641-80f6-97c8a7672187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c6f00d24905449c887b1a2f8149b4ff'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:13.638859 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8da8ca20-983d-4641-80f6-97c8a7672187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c6f00d24905449c887b1a2f8149b4ff'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:13.647876 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 969/1919] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:13 2022] DELETE /identity/v3/users/0c6f00d24905449c887b1a2f8149b4ff => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:14.024075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:14.024075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:14.041365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5077f-5a74-4cda-a1e3-e5c367c3f1dc admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:14.110542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:14.111173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:14.111456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:14.111769 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5 admin admin] PATH_INFO: `/v3/projects/f67b152f8283494eaffcb4fb056758ad` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:14.112882 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f67b152f8283494eaffcb4fb056758ad)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:14.114820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:14.162546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f67b152f8283494eaffcb4fb056758ad', 'request_id': 'req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:14.168244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f67b152f8283494eaffcb4fb056758ad', 'request_id': 'req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:14.178269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f67b152f8283494eaffcb4fb056758ad', 'request_id': 'req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:14.195082 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5 admin admin] The token cache is being invalidate because project f67b152f8283494eaffcb4fb056758ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:14.195376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:14.196090 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2b4bdd04-e1a4-4572-a8f7-f414d51c23b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:14.197560 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 951/1920] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:14 2022] DELETE /identity/v3/projects/f67b152f8283494eaffcb4fb056758ad => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:14.206436 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4322a66-7d42-4337-919f-22aa94e9951f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:14.206900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4322a66-7d42-4337-919f-22aa94e9951f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:14.206990 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4322a66-7d42-4337-919f-22aa94e9951f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:14.227150 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a4322a66-7d42-4337-919f-22aa94e9951f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:14.228379 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a4322a66-7d42-4337-919f-22aa94e9951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:14.294017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 970/1921] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:14.302158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e60e83-ad13-4258-94e7-d6a3abb33864 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:14.302531 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e60e83-ad13-4258-94e7-d6a3abb33864 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:14.302816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e60e83-ad13-4258-94e7-d6a3abb33864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:14.324623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-98e60e83-ad13-4258-94e7-d6a3abb33864 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:14.325856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-98e60e83-ad13-4258-94e7-d6a3abb33864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:14.333972 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 952/1922] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:14.338203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4322a66-7d42-4337-919f-22aa94e9951f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:14.347711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4322a66-7d42-4337-919f-22aa94e9951f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:14.352162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40812dc6-afc7-4fd9-9ac2-be1a31b544d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:14.352752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812dc6-afc7-4fd9-9ac2-be1a31b544d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:14.353028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812dc6-afc7-4fd9-9ac2-be1a31b544d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:14.353304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812dc6-afc7-4fd9-9ac2-be1a31b544d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:14.353942 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40812dc6-afc7-4fd9-9ac2-be1a31b544d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:29:14.354244 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40812dc6-afc7-4fd9-9ac2-be1a31b544d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:14.355842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40812dc6-afc7-4fd9-9ac2-be1a31b544d5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:14.358096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:14.369939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 971/1923] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:29:14.374021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e60e83-ad13-4258-94e7-d6a3abb33864 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:14.383059 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e60e83-ad13-4258-94e7-d6a3abb33864 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:14.387624 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15339344-079d-48e0-aea3-76aba5072355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:14.388218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15339344-079d-48e0-aea3-76aba5072355 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:14.388494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15339344-079d-48e0-aea3-76aba5072355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:14.388762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15339344-079d-48e0-aea3-76aba5072355 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:14.389900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15339344-079d-48e0-aea3-76aba5072355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:14.391817 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15339344-079d-48e0-aea3-76aba5072355 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:14.422364 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 953/1924] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:14 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:29:14.423649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40812dc6-afc7-4fd9-9ac2-be1a31b544d5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:14.433837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40812dc6-afc7-4fd9-9ac2-be1a31b544d5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:14.494545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6b117-ce06-4be3-a788-3a9193f453a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:14.495274 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6b117-ce06-4be3-a788-3a9193f453a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:14.495561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6b117-ce06-4be3-a788-3a9193f453a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:14.495872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6b117-ce06-4be3-a788-3a9193f453a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:14.496516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6b117-ce06-4be3-a788-3a9193f453a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:14.498420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6b117-ce06-4be3-a788-3a9193f453a3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:14.521293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 972/1925] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:14 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:14.525758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15339344-079d-48e0-aea3-76aba5072355 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:14.538263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15339344-079d-48e0-aea3-76aba5072355 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:14.543277 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedb775-3a78-4bd4-8019-f15c944d4f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:14.543923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedb775-3a78-4bd4-8019-f15c944d4f40 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:14.544206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedb775-3a78-4bd4-8019-f15c944d4f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:14.544514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedb775-3a78-4bd4-8019-f15c944d4f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:14.545163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedb775-3a78-4bd4-8019-f15c944d4f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:14.548307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedb775-3a78-4bd4-8019-f15c944d4f40 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:14.560558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 954/1926] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:14.565161 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6b117-ce06-4be3-a788-3a9193f453a3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:14.575139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6b117-ce06-4be3-a788-3a9193f453a3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:14.580522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dd900-9862-439b-8b4b-927c05898c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:14.581123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dd900-9862-439b-8b4b-927c05898c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:14.581400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dd900-9862-439b-8b4b-927c05898c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:14.581668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dd900-9862-439b-8b4b-927c05898c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:14.582340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dd900-9862-439b-8b4b-927c05898c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:14.583956 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dd900-9862-439b-8b4b-927c05898c06 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:14.596896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 973/1927] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:14.601330 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedb775-3a78-4bd4-8019-f15c944d4f40 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:14.612184 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedb775-3a78-4bd4-8019-f15c944d4f40 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:14.618167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fcc07-1dae-4706-83ee-c83f93b00ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:14.618989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fcc07-1dae-4706-83ee-c83f93b00ca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:14.619266 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fcc07-1dae-4706-83ee-c83f93b00ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:14.619655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fcc07-1dae-4706-83ee-c83f93b00ca0 admin admin] PATH_INFO: `/v3/projects/5911efa01bf1431cb07e76d057f90c93/users/ac87d3f72ba2412699f2384977fd1c85/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:14.635697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fcc07-1dae-4706-83ee-c83f93b00ca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5911efa01bf1431cb07e76d057f90c93, user_id=ac87d3f72ba2412699f2384977fd1c85, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:14.639201 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fcc07-1dae-4706-83ee-c83f93b00ca0 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:14.643980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:14.665068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 955/1928] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:14 2022] PUT /identity/v3/projects/5911efa01bf1431cb07e76d057f90c93/users/ac87d3f72ba2412699f2384977fd1c85/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:14.677158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c913696-97f6-4bfc-bd33-db66874e8a82 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:14.677659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c913696-97f6-4bfc-bd33-db66874e8a82 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:14.678104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c913696-97f6-4bfc-bd33-db66874e8a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:14.687512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fcc07-1dae-4706-83ee-c83f93b00ca0 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:14.689368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-577fcc07-1dae-4706-83ee-c83f93b00ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:14.723406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5c913696-97f6-4bfc-bd33-db66874e8a82 None None] MFA Rules not processed for user `ac87d3f72ba2412699f2384977fd1c85`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:14.725172 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5c913696-97f6-4bfc-bd33-db66874e8a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:14.764518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fcc07-1dae-4706-83ee-c83f93b00ca0 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:14.769771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6c095-a3a9-4341-99b6-f4b6d6e4f8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:14.770775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6c095-a3a9-4341-99b6-f4b6d6e4f8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:14.771069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6c095-a3a9-4341-99b6-f4b6d6e4f8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:14.771347 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6c095-a3a9-4341-99b6-f4b6d6e4f8be admin admin] PATH_INFO: `/v3/users/2d2fa12dc34d4ceeaa748e0e92aa9791` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:14.773044 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6c095-a3a9-4341-99b6-f4b6d6e4f8be admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d2fa12dc34d4ceeaa748e0e92aa9791)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:14.774945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6c095-a3a9-4341-99b6-f4b6d6e4f8be admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:14.807732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 974/1929] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:14.808664 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c913696-97f6-4bfc-bd33-db66874e8a82 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:14.826772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c913696-97f6-4bfc-bd33-db66874e8a82 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:14.833230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537f1e2-74f0-4557-8359-c3d6bb87e9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:14.833851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537f1e2-74f0-4557-8359-c3d6bb87e9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:14.834138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537f1e2-74f0-4557-8359-c3d6bb87e9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:14.834406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537f1e2-74f0-4557-8359-c3d6bb87e9c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:14.835104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537f1e2-74f0-4557-8359-c3d6bb87e9c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:14.838049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537f1e2-74f0-4557-8359-c3d6bb87e9c0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:14.844613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-fda6c095-a3a9-4341-99b6-f4b6d6e4f8be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d2fa12dc34d4ceeaa748e0e92aa9791'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:14.870789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-fda6c095-a3a9-4341-99b6-f4b6d6e4f8be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d2fa12dc34d4ceeaa748e0e92aa9791'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:14.890205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-fda6c095-a3a9-4341-99b6-f4b6d6e4f8be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d2fa12dc34d4ceeaa748e0e92aa9791'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:14.891289 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-fda6c095-a3a9-4341-99b6-f4b6d6e4f8be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d2fa12dc34d4ceeaa748e0e92aa9791'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:14.895933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 975/1930] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:14 2022] POST /identity/v3/projects => generated 369 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:14.904534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-fda6c095-a3a9-4341-99b6-f4b6d6e4f8be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d2fa12dc34d4ceeaa748e0e92aa9791'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:14.905934 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537f1e2-74f0-4557-8359-c3d6bb87e9c0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:14.908031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4537f1e2-74f0-4557-8359-c3d6bb87e9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:14.916246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 956/1931] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:14 2022] DELETE /identity/v3/users/2d2fa12dc34d4ceeaa748e0e92aa9791 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:14.919179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb68c4e-f1cc-4684-a580-6611d545515d None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:14.919469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb68c4e-f1cc-4684-a580-6611d545515d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:14.919703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb68c4e-f1cc-4684-a580-6611d545515d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:14.941625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-ddb68c4e-f1cc-4684-a580-6611d545515d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:14.942808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ddb68c4e-f1cc-4684-a580-6611d545515d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:14.970565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:14.992865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537f1e2-74f0-4557-8359-c3d6bb87e9c0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:15.001899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 957/1932] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:15.011738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb68c4e-f1cc-4684-a580-6611d545515d None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:15.013183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ddb68c4e-f1cc-4684-a580-6611d545515d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:15.026859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:15.054258 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c23fcc-9150-4aaa-8824-d579c25eccf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:15.055169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c23fcc-9150-4aaa-8824-d579c25eccf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:15.055450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c23fcc-9150-4aaa-8824-d579c25eccf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:15.055759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c23fcc-9150-4aaa-8824-d579c25eccf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:15.056390 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c23fcc-9150-4aaa-8824-d579c25eccf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:15.058332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c23fcc-9150-4aaa-8824-d579c25eccf4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:15.073237 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb68c4e-f1cc-4684-a580-6611d545515d None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:15.077128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b0fa3-a36f-4552-9f31-b3973fd009a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:15.077731 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b0fa3-a36f-4552-9f31-b3973fd009a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:15.078037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b0fa3-a36f-4552-9f31-b3973fd009a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:15.078405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b0fa3-a36f-4552-9f31-b3973fd009a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:15.081196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 976/1933] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:14 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:29:15.085376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c23fcc-9150-4aaa-8824-d579c25eccf4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:15.088149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b0fa3-a36f-4552-9f31-b3973fd009a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:15.089591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b0fa3-a36f-4552-9f31-b3973fd009a0 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:15.094921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c23fcc-9150-4aaa-8824-d579c25eccf4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:15.099336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 958/1934] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:29:15.099469 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f885c-24a3-4f4a-a245-81fb09e62c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:15.100065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f885c-24a3-4f4a-a245-81fb09e62c43 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:15.100344 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f885c-24a3-4f4a-a245-81fb09e62c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:15.100613 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f885c-24a3-4f4a-a245-81fb09e62c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:15.101246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f885c-24a3-4f4a-a245-81fb09e62c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:15.102820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f885c-24a3-4f4a-a245-81fb09e62c43 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:15.115654 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 977/1935] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:29:15.117757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b0fa3-a36f-4552-9f31-b3973fd009a0 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:15.128470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b0fa3-a36f-4552-9f31-b3973fd009a0 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:15.133832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ccab3-5c2b-4978-b558-7de57b7c9a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:15.134476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ccab3-5c2b-4978-b558-7de57b7c9a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:15.134761 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ccab3-5c2b-4978-b558-7de57b7c9a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:15.135039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ccab3-5c2b-4978-b558-7de57b7c9a0f admin admin] PATH_INFO: `/v3/projects/2953ed6a55134c5fa3886a6486f3faff/users/01dea4b0eaf945e7aad9a9302acf5f7b/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:15.151261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ccab3-5c2b-4978-b558-7de57b7c9a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2953ed6a55134c5fa3886a6486f3faff, user_id=01dea4b0eaf945e7aad9a9302acf5f7b, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:15.154513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ccab3-5c2b-4978-b558-7de57b7c9a0f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:15.169332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 959/1936] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:15 2022] PUT /identity/v3/projects/2953ed6a55134c5fa3886a6486f3faff/users/01dea4b0eaf945e7aad9a9302acf5f7b/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:15.174236 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f885c-24a3-4f4a-a245-81fb09e62c43 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:15.187822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f885c-24a3-4f4a-a245-81fb09e62c43 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:15.259193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d8771-4b80-472a-8af0-ebef7376f1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:15.259837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d8771-4b80-472a-8af0-ebef7376f1ce admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:15.260124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d8771-4b80-472a-8af0-ebef7376f1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:15.260403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d8771-4b80-472a-8af0-ebef7376f1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:15.261053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d8771-4b80-472a-8af0-ebef7376f1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:15.262655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d8771-4b80-472a-8af0-ebef7376f1ce admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:15.273223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 978/1937] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:29:15.277941 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ccab3-5c2b-4978-b558-7de57b7c9a0f admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:15.289893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ccab3-5c2b-4978-b558-7de57b7c9a0f admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:15.295086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14103461-2c12-4210-9633-a97feae6fb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:15.296637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14103461-2c12-4210-9633-a97feae6fb40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:15.296989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14103461-2c12-4210-9633-a97feae6fb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:15.297339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14103461-2c12-4210-9633-a97feae6fb40 admin admin] PATH_INFO: `/v3/projects/2953ed6a55134c5fa3886a6486f3faff/users/01dea4b0eaf945e7aad9a9302acf5f7b/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:15.299387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14103461-2c12-4210-9633-a97feae6fb40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2953ed6a55134c5fa3886a6486f3faff, user_id=01dea4b0eaf945e7aad9a9302acf5f7b, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:15.302206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14103461-2c12-4210-9633-a97feae6fb40 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:15.314433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 960/1938] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:15 2022] PUT /identity/v3/projects/2953ed6a55134c5fa3886a6486f3faff/users/01dea4b0eaf945e7aad9a9302acf5f7b/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:15.326102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f942-e12e-4fdb-a21c-3ed1fd8832c8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:15.326394 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f942-e12e-4fdb-a21c-3ed1fd8832c8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:15.326672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f942-e12e-4fdb-a21c-3ed1fd8832c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:15.355814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1162f942-e12e-4fdb-a21c-3ed1fd8832c8 None None] MFA Rules not processed for user `01dea4b0eaf945e7aad9a9302acf5f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:15.357074 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1162f942-e12e-4fdb-a21c-3ed1fd8832c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:15.386225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14103461-2c12-4210-9633-a97feae6fb40 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:15.387983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-14103461-2c12-4210-9633-a97feae6fb40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:15.414273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:15.436993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 979/1939] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:15.448142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162f942-e12e-4fdb-a21c-3ed1fd8832c8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:15.449545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1162f942-e12e-4fdb-a21c-3ed1fd8832c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:15.468628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14103461-2c12-4210-9633-a97feae6fb40 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:15.473433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:15.474216 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:15.474498 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:15.474773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] PATH_INFO: `/v3/projects/08fff01b75b04446bcf402130343bc77` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:15.476074 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08fff01b75b04446bcf402130343bc77)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:15.478013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:15.509793 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162f942-e12e-4fdb-a21c-3ed1fd8832c8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:15.513589 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184153d-dc6f-42af-8974-0d440d45d6f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:15.514190 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184153d-dc6f-42af-8974-0d440d45d6f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:15.514528 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184153d-dc6f-42af-8974-0d440d45d6f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:15.514801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184153d-dc6f-42af-8974-0d440d45d6f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:15.523778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184153d-dc6f-42af-8974-0d440d45d6f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:15.525181 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184153d-dc6f-42af-8974-0d440d45d6f0 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:15.526657 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08fff01b75b04446bcf402130343bc77', 'request_id': 'req-2c68da4d-d04f-45f9-9e4f-807911030f55'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:15.533051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08fff01b75b04446bcf402130343bc77', 'request_id': 'req-2c68da4d-d04f-45f9-9e4f-807911030f55'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:15.535610 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 980/1940] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:29:15.540836 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08fff01b75b04446bcf402130343bc77', 'request_id': 'req-2c68da4d-d04f-45f9-9e4f-807911030f55'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:15.552984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] The token cache is being invalidate because project 08fff01b75b04446bcf402130343bc77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:15.553273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:15.553951 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:15.555533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 961/1941] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:15 2022] DELETE /identity/v3/projects/08fff01b75b04446bcf402130343bc77 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:15.560515 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184153d-dc6f-42af-8974-0d440d45d6f0 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:15.571742 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184153d-dc6f-42af-8974-0d440d45d6f0 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:15.585325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:15.587049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:15.598564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:15.631141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe502b-577d-4760-a1e0-727617fb09d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:15.631840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe502b-577d-4760-a1e0-727617fb09d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:15.632117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe502b-577d-4760-a1e0-727617fb09d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:15.632392 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe502b-577d-4760-a1e0-727617fb09d3 admin admin] PATH_INFO: `/v3/users/7e2700131d204ee393a5602068a8bb4a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:15.633561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe502b-577d-4760-a1e0-727617fb09d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e2700131d204ee393a5602068a8bb4a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:15.635462 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe502b-577d-4760-a1e0-727617fb09d3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:15.647962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68da4d-d04f-45f9-9e4f-807911030f55 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:15.651729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44569782-a3ec-4f8f-80b8-d252c0e4bf3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:15.652341 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44569782-a3ec-4f8f-80b8-d252c0e4bf3e service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:15.652624 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44569782-a3ec-4f8f-80b8-d252c0e4bf3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:15.652898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44569782-a3ec-4f8f-80b8-d252c0e4bf3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:15.654649 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-44569782-a3ec-4f8f-80b8-d252c0e4bf3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:15.687907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2fe502b-577d-4760-a1e0-727617fb09d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e2700131d204ee393a5602068a8bb4a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:15.696471 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2fe502b-577d-4760-a1e0-727617fb09d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e2700131d204ee393a5602068a8bb4a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:15.715648 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2fe502b-577d-4760-a1e0-727617fb09d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e2700131d204ee393a5602068a8bb4a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:15.726369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2fe502b-577d-4760-a1e0-727617fb09d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e2700131d204ee393a5602068a8bb4a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:15.727018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2fe502b-577d-4760-a1e0-727617fb09d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e2700131d204ee393a5602068a8bb4a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:15.728738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44569782-a3ec-4f8f-80b8-d252c0e4bf3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:15.730814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44569782-a3ec-4f8f-80b8-d252c0e4bf3e service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:15.737626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 981/1942] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:15 2022] DELETE /identity/v3/users/7e2700131d204ee393a5602068a8bb4a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:15.741221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 962/1943] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:29:16.244174 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe502b-577d-4760-a1e0-727617fb09d3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:16.245785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e2fe502b-577d-4760-a1e0-727617fb09d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:16.262359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:16.330763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe502b-577d-4760-a1e0-727617fb09d3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:16.335299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456599e-2a5a-4217-81c9-fb3071b7d501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:16.335961 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456599e-2a5a-4217-81c9-fb3071b7d501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:16.336240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456599e-2a5a-4217-81c9-fb3071b7d501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:16.336509 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456599e-2a5a-4217-81c9-fb3071b7d501 admin admin] PATH_INFO: `/v3/projects/8d1fd16d8bf04f90bb348a26273d3f5a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:16.337632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456599e-2a5a-4217-81c9-fb3071b7d501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d1fd16d8bf04f90bb348a26273d3f5a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:16.339532 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456599e-2a5a-4217-81c9-fb3071b7d501 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:16.384677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6456599e-2a5a-4217-81c9-fb3071b7d501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d1fd16d8bf04f90bb348a26273d3f5a', 'request_id': 'req-6456599e-2a5a-4217-81c9-fb3071b7d501'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:16.392772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6456599e-2a5a-4217-81c9-fb3071b7d501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d1fd16d8bf04f90bb348a26273d3f5a', 'request_id': 'req-6456599e-2a5a-4217-81c9-fb3071b7d501'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:16.403809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6456599e-2a5a-4217-81c9-fb3071b7d501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d1fd16d8bf04f90bb348a26273d3f5a', 'request_id': 'req-6456599e-2a5a-4217-81c9-fb3071b7d501'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:16.416152 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6456599e-2a5a-4217-81c9-fb3071b7d501 admin admin] The token cache is being invalidate because project 8d1fd16d8bf04f90bb348a26273d3f5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:16.416562 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6456599e-2a5a-4217-81c9-fb3071b7d501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:16.417207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6456599e-2a5a-4217-81c9-fb3071b7d501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:16.418684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 982/1944] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:16 2022] DELETE /identity/v3/projects/8d1fd16d8bf04f90bb348a26273d3f5a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:16.424209 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4f0e3-fc99-44dd-8229-76682d7c66d7 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:16.424735 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4f0e3-fc99-44dd-8229-76682d7c66d7 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:16.424833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4f0e3-fc99-44dd-8229-76682d7c66d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:16.444769 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-08e4f0e3-fc99-44dd-8229-76682d7c66d7 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:16.445913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-08e4f0e3-fc99-44dd-8229-76682d7c66d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:16.504648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 963/1945] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:16.512718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6ee29-9a11-4a32-ba01-24bbdb2dae92 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:16.513011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6ee29-9a11-4a32-ba01-24bbdb2dae92 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:16.513289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6ee29-9a11-4a32-ba01-24bbdb2dae92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:16.532731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-88e6ee29-9a11-4a32-ba01-24bbdb2dae92 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:16.533944 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-88e6ee29-9a11-4a32-ba01-24bbdb2dae92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:16.541877 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 983/1946] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:16.546488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4f0e3-fc99-44dd-8229-76682d7c66d7 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:16.555954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4f0e3-fc99-44dd-8229-76682d7c66d7 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:16.560487 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b847aa-15c7-44ef-baab-06f125c1915d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:16.561092 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b847aa-15c7-44ef-baab-06f125c1915d admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:16.561984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b847aa-15c7-44ef-baab-06f125c1915d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:16.562254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b847aa-15c7-44ef-baab-06f125c1915d admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:16.562895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b847aa-15c7-44ef-baab-06f125c1915d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:29:16.563194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b847aa-15c7-44ef-baab-06f125c1915d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:16.564802 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b847aa-15c7-44ef-baab-06f125c1915d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:16.575979 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 964/1947] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:29:16.579959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6ee29-9a11-4a32-ba01-24bbdb2dae92 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:16.589038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6ee29-9a11-4a32-ba01-24bbdb2dae92 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:16.593825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5e351-45b7-4361-859e-422d7491c7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:16.594430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5e351-45b7-4361-859e-422d7491c7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:16.594734 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5e351-45b7-4361-859e-422d7491c7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:16.595011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5e351-45b7-4361-859e-422d7491c7e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:16.595726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5e351-45b7-4361-859e-422d7491c7e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:16.597664 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5e351-45b7-4361-859e-422d7491c7e1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:16.625559 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 984/1948] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:16 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:16.629425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b847aa-15c7-44ef-baab-06f125c1915d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:16.640351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b847aa-15c7-44ef-baab-06f125c1915d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:16.687265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:16.697604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a533d7d-0782-45c6-a801-38bbbe5a8612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:16.698149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a533d7d-0782-45c6-a801-38bbbe5a8612 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:16.698431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a533d7d-0782-45c6-a801-38bbbe5a8612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:16.698708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a533d7d-0782-45c6-a801-38bbbe5a8612 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:16.699348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a533d7d-0782-45c6-a801-38bbbe5a8612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:16.701393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a533d7d-0782-45c6-a801-38bbbe5a8612 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:16.722097 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 965/1949] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:16 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:16.726334 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5e351-45b7-4361-859e-422d7491c7e1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:16.736891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5e351-45b7-4361-859e-422d7491c7e1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:16.741507 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c31b97-445d-43c1-a9e7-a5fa6bb43c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:16.742097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c31b97-445d-43c1-a9e7-a5fa6bb43c78 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:16.742373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c31b97-445d-43c1-a9e7-a5fa6bb43c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:16.742641 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c31b97-445d-43c1-a9e7-a5fa6bb43c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:16.743279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c31b97-445d-43c1-a9e7-a5fa6bb43c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:16.744900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c31b97-445d-43c1-a9e7-a5fa6bb43c78 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:16.755026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 985/1950] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:16.758770 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a533d7d-0782-45c6-a801-38bbbe5a8612 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:16.768585 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a533d7d-0782-45c6-a801-38bbbe5a8612 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:16.773151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581be216-5c13-44cf-b045-9a9f96fee371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:16.773741 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581be216-5c13-44cf-b045-9a9f96fee371 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:16.774021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581be216-5c13-44cf-b045-9a9f96fee371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:16.774295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581be216-5c13-44cf-b045-9a9f96fee371 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:16.774932 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581be216-5c13-44cf-b045-9a9f96fee371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:16.776548 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581be216-5c13-44cf-b045-9a9f96fee371 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:16.787180 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 966/1951] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:16.791313 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c31b97-445d-43c1-a9e7-a5fa6bb43c78 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:16.801986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c31b97-445d-43c1-a9e7-a5fa6bb43c78 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:16.806634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429ce92-a0d4-4e89-84ef-72d0955ead57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:16.807279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429ce92-a0d4-4e89-84ef-72d0955ead57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:16.807558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429ce92-a0d4-4e89-84ef-72d0955ead57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:16.807864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429ce92-a0d4-4e89-84ef-72d0955ead57 admin admin] PATH_INFO: `/v3/projects/86448ad59ed94f15be8d6c1ad6a5eac8/users/f3b573d6bde14abd8c84d1863bc5cfb5/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:16.822052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b429ce92-a0d4-4e89-84ef-72d0955ead57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86448ad59ed94f15be8d6c1ad6a5eac8, user_id=f3b573d6bde14abd8c84d1863bc5cfb5, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:16.824759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b429ce92-a0d4-4e89-84ef-72d0955ead57 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:16.837454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 986/1952] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:16 2022] PUT /identity/v3/projects/86448ad59ed94f15be8d6c1ad6a5eac8/users/f3b573d6bde14abd8c84d1863bc5cfb5/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:16.849228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc360f0-484e-4415-bca3-0d820b343a12 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:16.849518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc360f0-484e-4415-bca3-0d820b343a12 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:16.849791 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc360f0-484e-4415-bca3-0d820b343a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:16.877338 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-6fc360f0-484e-4415-bca3-0d820b343a12 None None] MFA Rules not processed for user `f3b573d6bde14abd8c84d1863bc5cfb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:16.878543 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6fc360f0-484e-4415-bca3-0d820b343a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:16.944918 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 967/1953] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:16.955448 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429ce92-a0d4-4e89-84ef-72d0955ead57 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:16.957190 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b429ce92-a0d4-4e89-84ef-72d0955ead57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:16.998100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:17.016702 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429ce92-a0d4-4e89-84ef-72d0955ead57 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:17.026221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0b500-58c0-4c41-8afa-605c4cdd3d48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:17.026836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0b500-58c0-4c41-8afa-605c4cdd3d48 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:17.027119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0b500-58c0-4c41-8afa-605c4cdd3d48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:17.027388 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0b500-58c0-4c41-8afa-605c4cdd3d48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:17.037444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0b500-58c0-4c41-8afa-605c4cdd3d48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:17.038862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0b500-58c0-4c41-8afa-605c4cdd3d48 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:17.048359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 987/1954] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:19.409091 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc360f0-484e-4415-bca3-0d820b343a12 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:19.410783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6fc360f0-484e-4415-bca3-0d820b343a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:19.493999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc360f0-484e-4415-bca3-0d820b343a12 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:19.499335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e000425-d1f7-462b-91c3-e31673c4abfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:19.500162 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e000425-d1f7-462b-91c3-e31673c4abfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:19.500733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e000425-d1f7-462b-91c3-e31673c4abfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:19.500733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e000425-d1f7-462b-91c3-e31673c4abfc admin admin] PATH_INFO: `/v3/users/ac87d3f72ba2412699f2384977fd1c85` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:19.501692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e000425-d1f7-462b-91c3-e31673c4abfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac87d3f72ba2412699f2384977fd1c85)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:19.503695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e000425-d1f7-462b-91c3-e31673c4abfc admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:19.551278 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2e000425-d1f7-462b-91c3-e31673c4abfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac87d3f72ba2412699f2384977fd1c85'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:19.561809 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:19.564461 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2e000425-d1f7-462b-91c3-e31673c4abfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac87d3f72ba2412699f2384977fd1c85'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:19.572331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2e000425-d1f7-462b-91c3-e31673c4abfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac87d3f72ba2412699f2384977fd1c85'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:19.572981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2e000425-d1f7-462b-91c3-e31673c4abfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac87d3f72ba2412699f2384977fd1c85'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:19.581301 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2e000425-d1f7-462b-91c3-e31673c4abfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac87d3f72ba2412699f2384977fd1c85'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:19.587876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 968/1955] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:19 2022] DELETE /identity/v3/users/ac87d3f72ba2412699f2384977fd1c85 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:19.592391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95493d92-00cd-4b49-85b1-2a9818bbf44d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:19.592875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95493d92-00cd-4b49-85b1-2a9818bbf44d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:19.592975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95493d92-00cd-4b49-85b1-2a9818bbf44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:19.612458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-95493d92-00cd-4b49-85b1-2a9818bbf44d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:19.613677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-95493d92-00cd-4b49-85b1-2a9818bbf44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:19.677946 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 988/1956] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:19.688298 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e000425-d1f7-462b-91c3-e31673c4abfc admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:19.689741 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2e000425-d1f7-462b-91c3-e31673c4abfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:19.752383 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e000425-d1f7-462b-91c3-e31673c4abfc admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:19.755962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a03315-8410-447a-a5bb-186059e3eff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:19.756590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a03315-8410-447a-a5bb-186059e3eff3 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:19.756867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a03315-8410-447a-a5bb-186059e3eff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:19.757138 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a03315-8410-447a-a5bb-186059e3eff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:19.766869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a03315-8410-447a-a5bb-186059e3eff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:19.769359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a03315-8410-447a-a5bb-186059e3eff3 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:19.783787 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 969/1957] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:19.903245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95493d92-00cd-4b49-85b1-2a9818bbf44d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:19.905206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-95493d92-00cd-4b49-85b1-2a9818bbf44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:19.923171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95493d92-00cd-4b49-85b1-2a9818bbf44d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:19.928227 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec33752-a72f-4d96-85aa-f86f7f8b644c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:19.928868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec33752-a72f-4d96-85aa-f86f7f8b644c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:19.929146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec33752-a72f-4d96-85aa-f86f7f8b644c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:19.929416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec33752-a72f-4d96-85aa-f86f7f8b644c admin admin] PATH_INFO: `/v3/users/604bae081ccf44e5ba2ffab401dcd738` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:19.930582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec33752-a72f-4d96-85aa-f86f7f8b644c admin admin] RBAC: Authorizing `identity:delete_user(user_id=604bae081ccf44e5ba2ffab401dcd738)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:19.932516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec33752-a72f-4d96-85aa-f86f7f8b644c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:20.005324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1ec33752-a72f-4d96-85aa-f86f7f8b644c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '604bae081ccf44e5ba2ffab401dcd738'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.016270 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1ec33752-a72f-4d96-85aa-f86f7f8b644c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '604bae081ccf44e5ba2ffab401dcd738'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.032138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1ec33752-a72f-4d96-85aa-f86f7f8b644c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '604bae081ccf44e5ba2ffab401dcd738'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.041750 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1ec33752-a72f-4d96-85aa-f86f7f8b644c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '604bae081ccf44e5ba2ffab401dcd738'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.042460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1ec33752-a72f-4d96-85aa-f86f7f8b644c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '604bae081ccf44e5ba2ffab401dcd738'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.049443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 989/1958] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:19 2022] DELETE /identity/v3/users/604bae081ccf44e5ba2ffab401dcd738 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:20.053903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfa80f-b76f-4ece-929e-91c2d42dd87e None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:20.054196 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfa80f-b76f-4ece-929e-91c2d42dd87e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:20.054471 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfa80f-b76f-4ece-929e-91c2d42dd87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:20.099551 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-37dfa80f-b76f-4ece-929e-91c2d42dd87e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:20.106545 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-37dfa80f-b76f-4ece-929e-91c2d42dd87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:20.185015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 970/1959] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:20.199157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec33752-a72f-4d96-85aa-f86f7f8b644c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:20.201033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1ec33752-a72f-4d96-85aa-f86f7f8b644c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:20.263893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfa80f-b76f-4ece-929e-91c2d42dd87e None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:20.266091 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-37dfa80f-b76f-4ece-929e-91c2d42dd87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:20.278673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec33752-a72f-4d96-85aa-f86f7f8b644c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:20.283597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d85a3-9784-4a61-aa28-19c90d30af90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:20.284518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d85a3-9784-4a61-aa28-19c90d30af90 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:20.284865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d85a3-9784-4a61-aa28-19c90d30af90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:20.285221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d85a3-9784-4a61-aa28-19c90d30af90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:20.287936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfa80f-b76f-4ece-929e-91c2d42dd87e None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:20.295063 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:20.298082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d85a3-9784-4a61-aa28-19c90d30af90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:20.299739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d85a3-9784-4a61-aa28-19c90d30af90 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:20.319101 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 990/1960] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:20.321121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:20.323498 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:20.323817 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:20.324128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] PATH_INFO: `/v3/projects/5911efa01bf1431cb07e76d057f90c93` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:20.325465 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=5911efa01bf1431cb07e76d057f90c93)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:20.328585 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:20.397118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5911efa01bf1431cb07e76d057f90c93', 'request_id': 'req-f618c636-62df-4403-8b28-4ec5acb29eed'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.406561 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5911efa01bf1431cb07e76d057f90c93', 'request_id': 'req-f618c636-62df-4403-8b28-4ec5acb29eed'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.416247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5911efa01bf1431cb07e76d057f90c93', 'request_id': 'req-f618c636-62df-4403-8b28-4ec5acb29eed'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.430126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] The token cache is being invalidate because project 5911efa01bf1431cb07e76d057f90c93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:20.431074 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.431660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.433737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 971/1961] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:20 2022] DELETE /identity/v3/projects/5911efa01bf1431cb07e76d057f90c93 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:20.438392 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d85a3-9784-4a61-aa28-19c90d30af90 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:20.451055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d85a3-9784-4a61-aa28-19c90d30af90 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:20.527921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b379df8-80d0-4cf3-a3a0-0b658c255980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:20.528772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b379df8-80d0-4cf3-a3a0-0b658c255980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:20.529541 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b379df8-80d0-4cf3-a3a0-0b658c255980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:20.530863 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b379df8-80d0-4cf3-a3a0-0b658c255980 admin admin] PATH_INFO: `/v3/users/01dea4b0eaf945e7aad9a9302acf5f7b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:20.534459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b379df8-80d0-4cf3-a3a0-0b658c255980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01dea4b0eaf945e7aad9a9302acf5f7b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:20.542407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b379df8-80d0-4cf3-a3a0-0b658c255980 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:20.600458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:20.601984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:20.616977 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:20.622515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618c636-62df-4403-8b28-4ec5acb29eed admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:20.634704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59c2ca-9eb1-4a32-bbff-d2f2b0e1be78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:20.635902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59c2ca-9eb1-4a32-bbff-d2f2b0e1be78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:20.636293 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59c2ca-9eb1-4a32-bbff-d2f2b0e1be78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:20.636640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59c2ca-9eb1-4a32-bbff-d2f2b0e1be78 admin admin] PATH_INFO: `/v3/users/f3b573d6bde14abd8c84d1863bc5cfb5` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:20.638213 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59c2ca-9eb1-4a32-bbff-d2f2b0e1be78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b573d6bde14abd8c84d1863bc5cfb5)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:20.642176 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59c2ca-9eb1-4a32-bbff-d2f2b0e1be78 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:20.646186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b379df8-80d0-4cf3-a3a0-0b658c255980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01dea4b0eaf945e7aad9a9302acf5f7b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.659652 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b379df8-80d0-4cf3-a3a0-0b658c255980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01dea4b0eaf945e7aad9a9302acf5f7b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.674374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b379df8-80d0-4cf3-a3a0-0b658c255980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01dea4b0eaf945e7aad9a9302acf5f7b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.685883 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b379df8-80d0-4cf3-a3a0-0b658c255980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01dea4b0eaf945e7aad9a9302acf5f7b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.686891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b379df8-80d0-4cf3-a3a0-0b658c255980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01dea4b0eaf945e7aad9a9302acf5f7b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.698998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 991/1962] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:20 2022] DELETE /identity/v3/users/01dea4b0eaf945e7aad9a9302acf5f7b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:20.707901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6f59c2ca-9eb1-4a32-bbff-d2f2b0e1be78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b573d6bde14abd8c84d1863bc5cfb5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.722697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6f59c2ca-9eb1-4a32-bbff-d2f2b0e1be78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b573d6bde14abd8c84d1863bc5cfb5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.731393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6f59c2ca-9eb1-4a32-bbff-d2f2b0e1be78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b573d6bde14abd8c84d1863bc5cfb5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.732193 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6f59c2ca-9eb1-4a32-bbff-d2f2b0e1be78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b573d6bde14abd8c84d1863bc5cfb5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.740527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6f59c2ca-9eb1-4a32-bbff-d2f2b0e1be78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b573d6bde14abd8c84d1863bc5cfb5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:20.750555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 972/1963] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:20 2022] DELETE /identity/v3/users/f3b573d6bde14abd8c84d1863bc5cfb5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:20.755277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e170054-9e5e-4b40-8591-0921afd24da0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:20.755605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e170054-9e5e-4b40-8591-0921afd24da0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:20.755941 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e170054-9e5e-4b40-8591-0921afd24da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:20.783008 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1e170054-9e5e-4b40-8591-0921afd24da0 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:20.785382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1e170054-9e5e-4b40-8591-0921afd24da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:20.867924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 992/1964] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:20.878223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59c2ca-9eb1-4a32-bbff-d2f2b0e1be78 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:20.879744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6f59c2ca-9eb1-4a32-bbff-d2f2b0e1be78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:20.910248 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e170054-9e5e-4b40-8591-0921afd24da0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:20.916779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1e170054-9e5e-4b40-8591-0921afd24da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:20.929866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e170054-9e5e-4b40-8591-0921afd24da0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:20.935341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:20.936630 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:20.936986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:20.937523 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c admin admin] PATH_INFO: `/v3/projects/96222758254a4bd6971a94e0a7d058a4` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:20.938763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=96222758254a4bd6971a94e0a7d058a4)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:20.940718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:20.953916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:20.958578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59c2ca-9eb1-4a32-bbff-d2f2b0e1be78 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:20.964000 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989ad3a-9e90-4538-87d8-aaba552ab842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:20.965185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989ad3a-9e90-4538-87d8-aaba552ab842 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:20.965513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989ad3a-9e90-4538-87d8-aaba552ab842 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:20.965864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989ad3a-9e90-4538-87d8-aaba552ab842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:20.997764 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989ad3a-9e90-4538-87d8-aaba552ab842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:20.999901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989ad3a-9e90-4538-87d8-aaba552ab842 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:21.015189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 973/1965] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:21.017253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96222758254a4bd6971a94e0a7d058a4', 'request_id': 'req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.026128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96222758254a4bd6971a94e0a7d058a4', 'request_id': 'req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.041986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96222758254a4bd6971a94e0a7d058a4', 'request_id': 'req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.068941 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c admin admin] The token cache is being invalidate because project 96222758254a4bd6971a94e0a7d058a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:21.069431 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.070297 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4bd59d93-20b6-4ea7-9923-77fd752f4f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.071860 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 993/1966] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:20 2022] DELETE /identity/v3/projects/96222758254a4bd6971a94e0a7d058a4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:21.078945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6c109-37ee-475d-bcea-97a41337ff42 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.078945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6c109-37ee-475d-bcea-97a41337ff42 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.078945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6c109-37ee-475d-bcea-97a41337ff42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.098500 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-77f6c109-37ee-475d-bcea-97a41337ff42 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:21.099747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-77f6c109-37ee-475d-bcea-97a41337ff42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:21.172655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 974/1967] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:21.181510 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e8f37-369c-46fd-af26-d11aebba961c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.181800 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e8f37-369c-46fd-af26-d11aebba961c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.182082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e8f37-369c-46fd-af26-d11aebba961c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.202899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-819e8f37-369c-46fd-af26-d11aebba961c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:21.204148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-819e8f37-369c-46fd-af26-d11aebba961c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:21.212150 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 994/1968] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:21.216570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6c109-37ee-475d-bcea-97a41337ff42 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:21.228220 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6c109-37ee-475d-bcea-97a41337ff42 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:21.233453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014b8f4-3133-4dff-9009-d3331f67b602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:21.234188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014b8f4-3133-4dff-9009-d3331f67b602 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.234469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014b8f4-3133-4dff-9009-d3331f67b602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.234743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014b8f4-3133-4dff-9009-d3331f67b602 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.235403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014b8f4-3133-4dff-9009-d3331f67b602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:29:21.235728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014b8f4-3133-4dff-9009-d3331f67b602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:21.237329 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014b8f4-3133-4dff-9009-d3331f67b602 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:21.250592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 975/1969] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:29:21.255218 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e8f37-369c-46fd-af26-d11aebba961c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:21.265548 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e8f37-369c-46fd-af26-d11aebba961c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:21.270668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61230db-47fc-41b1-8086-67d7da995da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:21.271328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61230db-47fc-41b1-8086-67d7da995da5 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.271639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61230db-47fc-41b1-8086-67d7da995da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.271915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61230db-47fc-41b1-8086-67d7da995da5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.272581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61230db-47fc-41b1-8086-67d7da995da5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:21.274475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61230db-47fc-41b1-8086-67d7da995da5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:21.282885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014b8f4-3133-4dff-9009-d3331f67b602 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:21.284781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2014b8f4-3133-4dff-9009-d3331f67b602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:21.300507 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014b8f4-3133-4dff-9009-d3331f67b602 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:21.313761 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6215aeb-a564-4284-80cc-5c562c7f31cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:21.315270 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6215aeb-a564-4284-80cc-5c562c7f31cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.315531 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6215aeb-a564-4284-80cc-5c562c7f31cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.316157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 995/1970] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:21 2022] POST /identity/v3/projects => generated 361 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:21.317096 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6215aeb-a564-4284-80cc-5c562c7f31cf admin admin] PATH_INFO: `/v3/projects/2953ed6a55134c5fa3886a6486f3faff` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.317210 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61230db-47fc-41b1-8086-67d7da995da5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:21.322949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6215aeb-a564-4284-80cc-5c562c7f31cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2953ed6a55134c5fa3886a6486f3faff)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:21.325455 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6215aeb-a564-4284-80cc-5c562c7f31cf admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:21.326686 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:21.387783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e6215aeb-a564-4284-80cc-5c562c7f31cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2953ed6a55134c5fa3886a6486f3faff', 'request_id': 'req-e6215aeb-a564-4284-80cc-5c562c7f31cf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.395716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e6215aeb-a564-4284-80cc-5c562c7f31cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2953ed6a55134c5fa3886a6486f3faff', 'request_id': 'req-e6215aeb-a564-4284-80cc-5c562c7f31cf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.405181 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61230db-47fc-41b1-8086-67d7da995da5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:21.406309 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e6215aeb-a564-4284-80cc-5c562c7f31cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2953ed6a55134c5fa3886a6486f3faff', 'request_id': 'req-e6215aeb-a564-4284-80cc-5c562c7f31cf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.420700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e6215aeb-a564-4284-80cc-5c562c7f31cf admin admin] The token cache is being invalidate because project 2953ed6a55134c5fa3886a6486f3faff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:21.421329 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e6215aeb-a564-4284-80cc-5c562c7f31cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.421716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e6215aeb-a564-4284-80cc-5c562c7f31cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.423468 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 976/1971] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:21 2022] DELETE /identity/v3/projects/2953ed6a55134c5fa3886a6486f3faff => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:21.428673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fc104-04bc-4a02-a5d1-0d7ae28f359f None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.428986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fc104-04bc-4a02-a5d1-0d7ae28f359f None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.429264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fc104-04bc-4a02-a5d1-0d7ae28f359f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.451433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-dd7fc104-04bc-4a02-a5d1-0d7ae28f359f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:21.453030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-dd7fc104-04bc-4a02-a5d1-0d7ae28f359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:21.470037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b09f63-918e-4ce0-aaf6-e1438f03528b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:21.470640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b09f63-918e-4ce0-aaf6-e1438f03528b admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.471027 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b09f63-918e-4ce0-aaf6-e1438f03528b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.471310 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b09f63-918e-4ce0-aaf6-e1438f03528b admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.472146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b09f63-918e-4ce0-aaf6-e1438f03528b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:21.474233 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b09f63-918e-4ce0-aaf6-e1438f03528b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:21.506632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 996/1972] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:21 2022] POST /identity/v3/users => generated 281 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:21.506632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b09f63-918e-4ce0-aaf6-e1438f03528b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:21.506632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-51b09f63-918e-4ce0-aaf6-e1438f03528b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:21.519756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b09f63-918e-4ce0-aaf6-e1438f03528b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:21.524167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 977/1973] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:21.524457 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:21.525016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fc104-04bc-4a02-a5d1-0d7ae28f359f None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:21.525124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.525356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.525630 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] PATH_INFO: `/v3/projects/86448ad59ed94f15be8d6c1ad6a5eac8` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.526749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86448ad59ed94f15be8d6c1ad6a5eac8)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:21.528676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:21.530479 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:21.532247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-dd7fc104-04bc-4a02-a5d1-0d7ae28f359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:21.545142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fc104-04bc-4a02-a5d1-0d7ae28f359f None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:21.549824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3fea2-0f0f-4015-b506-1c4f4f091d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:21.550421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3fea2-0f0f-4015-b506-1c4f4f091d05 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.550708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3fea2-0f0f-4015-b506-1c4f4f091d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.550997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3fea2-0f0f-4015-b506-1c4f4f091d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.551672 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3fea2-0f0f-4015-b506-1c4f4f091d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:21.553281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3fea2-0f0f-4015-b506-1c4f4f091d05 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:21.564651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 978/1974] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:21.565841 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295f5bb-b91e-4f58-8f95-e6747dd5cdde None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.566127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295f5bb-b91e-4f58-8f95-e6747dd5cdde None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.566400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295f5bb-b91e-4f58-8f95-e6747dd5cdde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.581069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86448ad59ed94f15be8d6c1ad6a5eac8', 'request_id': 'req-2092dd09-0a3b-4d0e-8b32-c14c778e5133'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.586100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-e295f5bb-b91e-4f58-8f95-e6747dd5cdde None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:21.586789 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86448ad59ed94f15be8d6c1ad6a5eac8', 'request_id': 'req-2092dd09-0a3b-4d0e-8b32-c14c778e5133'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.587285 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e295f5bb-b91e-4f58-8f95-e6747dd5cdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:21.594858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86448ad59ed94f15be8d6c1ad6a5eac8', 'request_id': 'req-2092dd09-0a3b-4d0e-8b32-c14c778e5133'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.606995 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] The token cache is being invalidate because project 86448ad59ed94f15be8d6c1ad6a5eac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:21.607293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.608643 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:21.610164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 997/1975] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:21 2022] DELETE /identity/v3/projects/86448ad59ed94f15be8d6c1ad6a5eac8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:21.612337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:21.614198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:21.655540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 979/1976] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:21.655540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb06459-4fbc-401e-a040-022ad77e7948 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.655540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb06459-4fbc-401e-a040-022ad77e7948 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.655540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb06459-4fbc-401e-a040-022ad77e7948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.675219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-4fb06459-4fbc-401e-a040-022ad77e7948 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:21.676452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4fb06459-4fbc-401e-a040-022ad77e7948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:21.678625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092dd09-0a3b-4d0e-8b32-c14c778e5133 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:21.683188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad5eee-6477-410e-8838-25b03026cb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:21.683824 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad5eee-6477-410e-8838-25b03026cb50 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.684104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad5eee-6477-410e-8838-25b03026cb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.684331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 980/1977] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:21.684446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad5eee-6477-410e-8838-25b03026cb50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.685025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad5eee-6477-410e-8838-25b03026cb50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:21.685125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb06459-4fbc-401e-a040-022ad77e7948 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:21.687114 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad5eee-6477-410e-8838-25b03026cb50 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:21.694918 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb06459-4fbc-401e-a040-022ad77e7948 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:21.697620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 998/1978] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:21.698687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0f99c-a098-4037-a2f2-2caad24ee530 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.698974 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0f99c-a098-4037-a2f2-2caad24ee530 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.699254 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0f99c-a098-4037-a2f2-2caad24ee530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.699778 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf6825-2f55-4076-bbc8-40536e8ee46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:21.700405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf6825-2f55-4076-bbc8-40536e8ee46e admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.700643 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf6825-2f55-4076-bbc8-40536e8ee46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.700913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf6825-2f55-4076-bbc8-40536e8ee46e admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.701602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf6825-2f55-4076-bbc8-40536e8ee46e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:29:21.701934 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf6825-2f55-4076-bbc8-40536e8ee46e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:21.703506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf6825-2f55-4076-bbc8-40536e8ee46e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:21.713128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:21.718921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e1c0f99c-a098-4037-a2f2-2caad24ee530 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:21.720343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 981/1979] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:29:21.720453 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e1c0f99c-a098-4037-a2f2-2caad24ee530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:21.721414 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf6825-2f55-4076-bbc8-40536e8ee46e admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:21.728979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 999/1980] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:21.729735 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0f99c-a098-4037-a2f2-2caad24ee530 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:21.736387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf6825-2f55-4076-bbc8-40536e8ee46e admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:21.739617 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0f99c-a098-4037-a2f2-2caad24ee530 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:21.741088 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda5c45-9986-495e-afb0-3512163e55cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:21.741961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda5c45-9986-495e-afb0-3512163e55cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.742241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda5c45-9986-495e-afb0-3512163e55cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.742512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda5c45-9986-495e-afb0-3512163e55cf admin admin] PATH_INFO: `/v3/projects/225b6c3b2fa04864880e31300a37f82a/users/faf8fe96bfef4118a936d454e921bf6d/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.744938 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b302f7e-5ec9-42cb-82fb-68731f579ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:21.745530 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b302f7e-5ec9-42cb-82fb-68731f579ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.745857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b302f7e-5ec9-42cb-82fb-68731f579ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.746132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b302f7e-5ec9-42cb-82fb-68731f579ac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.746870 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b302f7e-5ec9-42cb-82fb-68731f579ac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:21.748905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b302f7e-5ec9-42cb-82fb-68731f579ac3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:21.758176 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda5c45-9986-495e-afb0-3512163e55cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=225b6c3b2fa04864880e31300a37f82a, user_id=faf8fe96bfef4118a936d454e921bf6d, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:21.761791 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda5c45-9986-495e-afb0-3512163e55cf admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:21.775633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 982/1981] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:21 2022] PUT /identity/v3/projects/225b6c3b2fa04864880e31300a37f82a/users/faf8fe96bfef4118a936d454e921bf6d/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:21.776431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda5c45-9986-495e-afb0-3512163e55cf admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:21.781407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1000/1982] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:21 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:21.782091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b302f7e-5ec9-42cb-82fb-68731f579ac3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:21.849564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda5c45-9986-495e-afb0-3512163e55cf admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:21.852364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:21.856449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b302f7e-5ec9-42cb-82fb-68731f579ac3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:21.858232 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbea80-4111-41d8-894d-0e2e9ebab75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:21.858603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbea80-4111-41d8-894d-0e2e9ebab75a admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.859030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbea80-4111-41d8-894d-0e2e9ebab75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.859417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbea80-4111-41d8-894d-0e2e9ebab75a admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.860503 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbea80-4111-41d8-894d-0e2e9ebab75a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:29:21.861358 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbea80-4111-41d8-894d-0e2e9ebab75a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:21.863864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbea80-4111-41d8-894d-0e2e9ebab75a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:21.863980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012091f-31dd-49f1-9eac-1db5e783cb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:21.864701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012091f-31dd-49f1-9eac-1db5e783cb57 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.865055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012091f-31dd-49f1-9eac-1db5e783cb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.865706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012091f-31dd-49f1-9eac-1db5e783cb57 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.866412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012091f-31dd-49f1-9eac-1db5e783cb57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:21.868848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012091f-31dd-49f1-9eac-1db5e783cb57 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:21.879873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 983/1983] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:29:21.881235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79f090-716f-4077-b61a-4387bbd0286b None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.881519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79f090-716f-4077-b61a-4387bbd0286b None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.881789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79f090-716f-4077-b61a-4387bbd0286b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.897552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1001/1984] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:21 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:29:21.898469 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012091f-31dd-49f1-9eac-1db5e783cb57 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:21.910570 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012091f-31dd-49f1-9eac-1db5e783cb57 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:21.920588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-9d79f090-716f-4077-b61a-4387bbd0286b None None] MFA Rules not processed for user `faf8fe96bfef4118a936d454e921bf6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:21.921930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9d79f090-716f-4077-b61a-4387bbd0286b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:21.927198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb25e2e-6b0c-4c0b-9842-5f5c44e58620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:21.927856 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb25e2e-6b0c-4c0b-9842-5f5c44e58620 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:21.928146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb25e2e-6b0c-4c0b-9842-5f5c44e58620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:21.928418 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb25e2e-6b0c-4c0b-9842-5f5c44e58620 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:21.929049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb25e2e-6b0c-4c0b-9842-5f5c44e58620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:21.930959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb25e2e-6b0c-4c0b-9842-5f5c44e58620 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:21.965711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1002/1985] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:21 2022] POST /identity/v3/projects => generated 369 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:21.965711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb25e2e-6b0c-4c0b-9842-5f5c44e58620 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.002325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 984/1986] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:22.006609 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79f090-716f-4077-b61a-4387bbd0286b None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.016659 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79f090-716f-4077-b61a-4387bbd0286b None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.053663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb25e2e-6b0c-4c0b-9842-5f5c44e58620 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.058486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080edae2-9736-4922-90f4-3e27842227fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.059131 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080edae2-9736-4922-90f4-3e27842227fb admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.059414 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080edae2-9736-4922-90f4-3e27842227fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.059718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080edae2-9736-4922-90f4-3e27842227fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.060357 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080edae2-9736-4922-90f4-3e27842227fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:22.062628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080edae2-9736-4922-90f4-3e27842227fb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:22.075713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1003/1987] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:22.075713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080edae2-9736-4922-90f4-3e27842227fb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.076571 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-080edae2-9736-4922-90f4-3e27842227fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:22.086514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d67d2-8f09-4cec-a981-49b6f018b4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.087275 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d67d2-8f09-4cec-a981-49b6f018b4ea admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.087557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d67d2-8f09-4cec-a981-49b6f018b4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.087859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d67d2-8f09-4cec-a981-49b6f018b4ea admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.088708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d67d2-8f09-4cec-a981-49b6f018b4ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:22.091135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d67d2-8f09-4cec-a981-49b6f018b4ea admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:22.099978 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:22.116893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 985/1988] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:22 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:22.120928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d67d2-8f09-4cec-a981-49b6f018b4ea admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.132450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d67d2-8f09-4cec-a981-49b6f018b4ea admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.137702 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f0c6d-e3f8-4091-89d6-ef9d2fe2b37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.138325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f0c6d-e3f8-4091-89d6-ef9d2fe2b37f admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.138604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f0c6d-e3f8-4091-89d6-ef9d2fe2b37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.138875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f0c6d-e3f8-4091-89d6-ef9d2fe2b37f admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.140188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f0c6d-e3f8-4091-89d6-ef9d2fe2b37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:22.141920 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f0c6d-e3f8-4091-89d6-ef9d2fe2b37f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:22.159557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 986/1989] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:22.164757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f0c6d-e3f8-4091-89d6-ef9d2fe2b37f admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.185177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f0c6d-e3f8-4091-89d6-ef9d2fe2b37f admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.187596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080edae2-9736-4922-90f4-3e27842227fb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.196259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee863b-9929-464c-a2a6-85b01c4862fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.196402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d4377-2966-4596-9ce2-3983c973e002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.197073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d4377-2966-4596-9ce2-3983c973e002 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.197464 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d4377-2966-4596-9ce2-3983c973e002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.197784 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee863b-9929-464c-a2a6-85b01c4862fb service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.197924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d4377-2966-4596-9ce2-3983c973e002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.198498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee863b-9929-464c-a2a6-85b01c4862fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.198872 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d4377-2966-4596-9ce2-3983c973e002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:22.199200 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee863b-9929-464c-a2a6-85b01c4862fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.200822 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d4377-2966-4596-9ce2-3983c973e002 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:22.217469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 987/1990] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:22.218682 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d4377-2966-4596-9ce2-3983c973e002 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.229262 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d4377-2966-4596-9ce2-3983c973e002 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.234302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175e562-7e0c-4824-8426-27fae8310677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.235047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175e562-7e0c-4824-8426-27fae8310677 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.235400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175e562-7e0c-4824-8426-27fae8310677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.235777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175e562-7e0c-4824-8426-27fae8310677 admin admin] PATH_INFO: `/v3/projects/a85123eaecc84a7dacd2ac22244a0088/users/aba748f11c554574b8c084b118700e1d/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.254346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175e562-7e0c-4824-8426-27fae8310677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85123eaecc84a7dacd2ac22244a0088, user_id=aba748f11c554574b8c084b118700e1d, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:22.257304 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175e562-7e0c-4824-8426-27fae8310677 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:22.281044 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 988/1991] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:22 2022] PUT /identity/v3/projects/a85123eaecc84a7dacd2ac22244a0088/users/aba748f11c554574b8c084b118700e1d/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:22.281044 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175e562-7e0c-4824-8426-27fae8310677 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.296611 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee863b-9929-464c-a2a6-85b01c4862fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:22.298232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee863b-9929-464c-a2a6-85b01c4862fb service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:22.307305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1004/1992] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:22.308739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b76680-3e23-40f0-8941-55dd460f45bb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.309104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b76680-3e23-40f0-8941-55dd460f45bb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.309542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b76680-3e23-40f0-8941-55dd460f45bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.346083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-25b76680-3e23-40f0-8941-55dd460f45bb None None] MFA Rules not processed for user `aba748f11c554574b8c084b118700e1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:22.347326 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-25b76680-3e23-40f0-8941-55dd460f45bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:22.351436 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175e562-7e0c-4824-8426-27fae8310677 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.351836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:22.358856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4042f-9252-4e55-98c0-7c12cc317881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.359475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4042f-9252-4e55-98c0-7c12cc317881 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.359784 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4042f-9252-4e55-98c0-7c12cc317881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.360095 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4042f-9252-4e55-98c0-7c12cc317881 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.361013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee4042f-9252-4e55-98c0-7c12cc317881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:22.363063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee4042f-9252-4e55-98c0-7c12cc317881 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:22.375251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 989/1993] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:22.375251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4042f-9252-4e55-98c0-7c12cc317881 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.385579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4042f-9252-4e55-98c0-7c12cc317881 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.390446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe61df-13a0-4f59-be61-6e32b1aceddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.391132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe61df-13a0-4f59-be61-6e32b1aceddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.391634 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe61df-13a0-4f59-be61-6e32b1aceddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.391978 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe61df-13a0-4f59-be61-6e32b1aceddd admin admin] PATH_INFO: `/v3/users/faf8fe96bfef4118a936d454e921bf6d` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.393186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe61df-13a0-4f59-be61-6e32b1aceddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=faf8fe96bfef4118a936d454e921bf6d)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:22.395168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe61df-13a0-4f59-be61-6e32b1aceddd admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:22.426227 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1005/1994] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:22.427419 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b76680-3e23-40f0-8941-55dd460f45bb None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.440464 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b76680-3e23-40f0-8941-55dd460f45bb None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.445013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b036b8-19d5-4726-b6a9-c51ea895afb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.445662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b036b8-19d5-4726-b6a9-c51ea895afb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.445970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b036b8-19d5-4726-b6a9-c51ea895afb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.446247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b036b8-19d5-4726-b6a9-c51ea895afb3 admin admin] PATH_INFO: `/v3/projects/b0032c2dc74d42f6894f198c0918e0b9/users/2c2809d11db846908573b871b4200b77/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.448594 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e9fe61df-13a0-4f59-be61-6e32b1aceddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faf8fe96bfef4118a936d454e921bf6d'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:22.460842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b036b8-19d5-4726-b6a9-c51ea895afb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0032c2dc74d42f6894f198c0918e0b9, user_id=2c2809d11db846908573b871b4200b77, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:22.462318 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e9fe61df-13a0-4f59-be61-6e32b1aceddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faf8fe96bfef4118a936d454e921bf6d'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:22.463485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b036b8-19d5-4726-b6a9-c51ea895afb3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:22.472393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e9fe61df-13a0-4f59-be61-6e32b1aceddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faf8fe96bfef4118a936d454e921bf6d'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:22.473226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e9fe61df-13a0-4f59-be61-6e32b1aceddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faf8fe96bfef4118a936d454e921bf6d'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:22.480045 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1006/1995] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:22 2022] PUT /identity/v3/projects/b0032c2dc74d42f6894f198c0918e0b9/users/2c2809d11db846908573b871b4200b77/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:22.481104 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e9fe61df-13a0-4f59-be61-6e32b1aceddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faf8fe96bfef4118a936d454e921bf6d'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:22.488293 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 990/1996] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:22 2022] DELETE /identity/v3/users/faf8fe96bfef4118a936d454e921bf6d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:22.488784 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b036b8-19d5-4726-b6a9-c51ea895afb3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.490537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-94b036b8-19d5-4726-b6a9-c51ea895afb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:22.494714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9437e97-3362-48a9-a139-3fa4b620288e None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.494999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9437e97-3362-48a9-a139-3fa4b620288e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.495312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9437e97-3362-48a9-a139-3fa4b620288e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.516285 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-e9437e97-3362-48a9-a139-3fa4b620288e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:22.517996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e9437e97-3362-48a9-a139-3fa4b620288e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:22.523267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:22.571262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b036b8-19d5-4726-b6a9-c51ea895afb3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.584996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 991/1997] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:29:22.585876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da83dad-7480-45a3-9c6e-3c15d2974069 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.586323 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da83dad-7480-45a3-9c6e-3c15d2974069 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.586643 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da83dad-7480-45a3-9c6e-3c15d2974069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.618642 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-2da83dad-7480-45a3-9c6e-3c15d2974069 None None] MFA Rules not processed for user `2c2809d11db846908573b871b4200b77`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:22.619921 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2da83dad-7480-45a3-9c6e-3c15d2974069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:22.649590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fe585-6785-4a86-82fb-6828bd9a71af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.650273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fe585-6785-4a86-82fb-6828bd9a71af admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.650563 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fe585-6785-4a86-82fb-6828bd9a71af admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.650978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fe585-6785-4a86-82fb-6828bd9a71af admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.651655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fe585-6785-4a86-82fb-6828bd9a71af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:22.653674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fe585-6785-4a86-82fb-6828bd9a71af admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:22.691854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1007/1998] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:22 2022] POST /identity/v3/projects => generated 389 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:22.691854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fe585-6785-4a86-82fb-6828bd9a71af admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.695095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0f2fe585-6785-4a86-82fb-6828bd9a71af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:22.713366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:22.716851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 992/1999] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:22.718301 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da83dad-7480-45a3-9c6e-3c15d2974069 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.731970 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da83dad-7480-45a3-9c6e-3c15d2974069 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.772416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fe585-6785-4a86-82fb-6828bd9a71af admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.778620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff19c7-8123-4062-81e8-0268a1966be4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.779272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff19c7-8123-4062-81e8-0268a1966be4 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.779610 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff19c7-8123-4062-81e8-0268a1966be4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.781105 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff19c7-8123-4062-81e8-0268a1966be4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.800493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dc5b1-930c-4890-9e66-8b76503cc6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.801393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dc5b1-930c-4890-9e66-8b76503cc6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.801678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dc5b1-930c-4890-9e66-8b76503cc6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.801978 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dc5b1-930c-4890-9e66-8b76503cc6d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.803462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63dc5b1-930c-4890-9e66-8b76503cc6d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:22.805617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63dc5b1-930c-4890-9e66-8b76503cc6d4 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:22.843361 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 993/2000] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:22 2022] POST /identity/v3/users => generated 301 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:22.843361 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dc5b1-930c-4890-9e66-8b76503cc6d4 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.845332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d63dc5b1-930c-4890-9e66-8b76503cc6d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:22.865396 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff19c7-8123-4062-81e8-0268a1966be4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:22.867111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff19c7-8123-4062-81e8-0268a1966be4 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:22.877458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1008/2001] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:22.878566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff19c7-8123-4062-81e8-0268a1966be4 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.888326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:22.889165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff19c7-8123-4062-81e8-0268a1966be4 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.895074 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daaffc8-a4b6-4039-ba19-a442867155f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.895876 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daaffc8-a4b6-4039-ba19-a442867155f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.896356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daaffc8-a4b6-4039-ba19-a442867155f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.896825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daaffc8-a4b6-4039-ba19-a442867155f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.897540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daaffc8-a4b6-4039-ba19-a442867155f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:22.899209 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daaffc8-a4b6-4039-ba19-a442867155f0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:22.910407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1009/2002] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:22.914606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daaffc8-a4b6-4039-ba19-a442867155f0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.923485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dc5b1-930c-4890-9e66-8b76503cc6d4 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.929370 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271722a-0477-4e0e-abaa-2ba1b6dda5d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61407406ef242a78030f79532fe2247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.929903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daaffc8-a4b6-4039-ba19-a442867155f0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:22.930012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271722a-0477-4e0e-abaa-2ba1b6dda5d2 service glance] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.930491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271722a-0477-4e0e-abaa-2ba1b6dda5d2 service glance] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.930604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271722a-0477-4e0e-abaa-2ba1b6dda5d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.935649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae85908-1bb8-4fc7-8a77-66c2488952fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:22.936509 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae85908-1bb8-4fc7-8a77-66c2488952fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:22.937064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae85908-1bb8-4fc7-8a77-66c2488952fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:22.937202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae85908-1bb8-4fc7-8a77-66c2488952fa admin admin] PATH_INFO: `/v3/projects/e9127c63f3f446e4b43f957ce9801c0d/users/d83716aed9574798bc097b8b737aa4c3/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:22.955836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae85908-1bb8-4fc7-8a77-66c2488952fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9127c63f3f446e4b43f957ce9801c0d, user_id=d83716aed9574798bc097b8b737aa4c3, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:22.959528 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae85908-1bb8-4fc7-8a77-66c2488952fa admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:22.974774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1010/2003] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:22 2022] PUT /identity/v3/projects/e9127c63f3f446e4b43f957ce9801c0d/users/d83716aed9574798bc097b8b737aa4c3/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:22.979147 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae85908-1bb8-4fc7-8a77-66c2488952fa admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:22.997955 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271722a-0477-4e0e-abaa-2ba1b6dda5d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:23.003923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271722a-0477-4e0e-abaa-2ba1b6dda5d2 service glance] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:23.013886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 994/2004] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:23.059665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae85908-1bb8-4fc7-8a77-66c2488952fa admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:23.067142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2858de0-2e85-440a-8217-140c5897a430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:23.068034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2858de0-2e85-440a-8217-140c5897a430 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:23.068343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2858de0-2e85-440a-8217-140c5897a430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:23.068626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2858de0-2e85-440a-8217-140c5897a430 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:23.069586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2858de0-2e85-440a-8217-140c5897a430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:23.071264 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2858de0-2e85-440a-8217-140c5897a430 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:23.087606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1011/2005] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:29:23.092923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271722a-0477-4e0e-abaa-2ba1b6dda5d2 service glance] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:23.114431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271722a-0477-4e0e-abaa-2ba1b6dda5d2 service glance] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:23.120336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af08e1-10de-44d6-990d-162b4744a4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:23.121068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af08e1-10de-44d6-990d-162b4744a4d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:23.121354 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af08e1-10de-44d6-990d-162b4744a4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:23.121615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af08e1-10de-44d6-990d-162b4744a4d7 admin admin] PATH_INFO: `/v3/projects/e9127c63f3f446e4b43f957ce9801c0d/users/d83716aed9574798bc097b8b737aa4c3/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:23.123915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af08e1-10de-44d6-990d-162b4744a4d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9127c63f3f446e4b43f957ce9801c0d, user_id=d83716aed9574798bc097b8b737aa4c3, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:23.127866 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af08e1-10de-44d6-990d-162b4744a4d7 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:23.148185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 995/2006] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:23 2022] PUT /identity/v3/projects/e9127c63f3f446e4b43f957ce9801c0d/users/d83716aed9574798bc097b8b737aa4c3/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:23.162244 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989e7e1-273e-4dc3-b518-f090052596e7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:23.162924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989e7e1-273e-4dc3-b518-f090052596e7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:23.163466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989e7e1-273e-4dc3-b518-f090052596e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:23.215782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d989e7e1-273e-4dc3-b518-f090052596e7 None None] MFA Rules not processed for user `d83716aed9574798bc097b8b737aa4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:23.217667 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d989e7e1-273e-4dc3-b518-f090052596e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:23.234788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:23.388733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af08e1-10de-44d6-990d-162b4744a4d7 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:23.390087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1012/2007] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:23 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:23.391343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b8af08e1-10de-44d6-990d-162b4744a4d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:23.551240 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af08e1-10de-44d6-990d-162b4744a4d7 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:23.558456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea0ad7-a1d5-40dd-b487-95a314e82475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:23.560171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea0ad7-a1d5-40dd-b487-95a314e82475 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:23.560685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea0ad7-a1d5-40dd-b487-95a314e82475 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:23.561802 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea0ad7-a1d5-40dd-b487-95a314e82475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:23.571264 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d989e7e1-273e-4dc3-b518-f090052596e7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:23.572926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d989e7e1-273e-4dc3-b518-f090052596e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:23.586772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea0ad7-a1d5-40dd-b487-95a314e82475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:23.589610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea0ad7-a1d5-40dd-b487-95a314e82475 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:23.628391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 996/2008] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:23 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:23.752941 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d989e7e1-273e-4dc3-b518-f090052596e7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:23.758790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:23.759769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:23.760155 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:23.760532 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] PATH_INFO: `/v3/projects/225b6c3b2fa04864880e31300a37f82a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:23.761773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=225b6c3b2fa04864880e31300a37f82a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:23.763831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:23.844740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '225b6c3b2fa04864880e31300a37f82a', 'request_id': 'req-2be40d95-4933-4ffb-9eca-041b8231c875'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:23.851363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '225b6c3b2fa04864880e31300a37f82a', 'request_id': 'req-2be40d95-4933-4ffb-9eca-041b8231c875'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:23.866334 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '225b6c3b2fa04864880e31300a37f82a', 'request_id': 'req-2be40d95-4933-4ffb-9eca-041b8231c875'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:23.882671 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] The token cache is being invalidate because project 225b6c3b2fa04864880e31300a37f82a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:23.882973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:23.883684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:23.885730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1013/2009] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:23 2022] DELETE /identity/v3/projects/225b6c3b2fa04864880e31300a37f82a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:23.890814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddf83b-9040-438f-b049-d904603afaf4 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:23.891102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddf83b-9040-438f-b049-d904603afaf4 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:23.891378 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddf83b-9040-438f-b049-d904603afaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:23.918062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-0bddf83b-9040-438f-b049-d904603afaf4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:23.919304 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-0bddf83b-9040-438f-b049-d904603afaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:23.981080 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:23.983213 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:24.008292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 997/2010] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:29:24.016717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848ef79-bc73-4f82-860c-98a86a8f77a2 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:24.017011 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848ef79-bc73-4f82-860c-98a86a8f77a2 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:24.017292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848ef79-bc73-4f82-860c-98a86a8f77a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:24.036710 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:24.038174 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-1848ef79-bc73-4f82-860c-98a86a8f77a2 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:24.039391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1848ef79-bc73-4f82-860c-98a86a8f77a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:24.051660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 998/2011] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:24.056029 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848ef79-bc73-4f82-860c-98a86a8f77a2 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:24.068454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be40d95-4933-4ffb-9eca-041b8231c875 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:24.069282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848ef79-bc73-4f82-860c-98a86a8f77a2 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:24.074288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641512e-9608-4d4c-a000-fce3a12ac2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:24.074485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d71f4f-55fe-48cb-a169-57945f8e6e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:24.074904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641512e-9608-4d4c-a000-fce3a12ac2ad admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:24.075134 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641512e-9608-4d4c-a000-fce3a12ac2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:24.075247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d71f4f-55fe-48cb-a169-57945f8e6e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:24.075400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641512e-9608-4d4c-a000-fce3a12ac2ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:24.075493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d71f4f-55fe-48cb-a169-57945f8e6e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:24.075802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d71f4f-55fe-48cb-a169-57945f8e6e98 admin admin] PATH_INFO: `/v3/users/aba748f11c554574b8c084b118700e1d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:24.076108 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641512e-9608-4d4c-a000-fce3a12ac2ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:29:24.076436 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641512e-9608-4d4c-a000-fce3a12ac2ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:24.076981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d71f4f-55fe-48cb-a169-57945f8e6e98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba748f11c554574b8c084b118700e1d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:24.078160 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641512e-9608-4d4c-a000-fce3a12ac2ad admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:24.079394 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d71f4f-55fe-48cb-a169-57945f8e6e98 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:24.080930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:24.096068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 999/2012] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:29:24.097680 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641512e-9608-4d4c-a000-fce3a12ac2ad admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:24.113787 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641512e-9608-4d4c-a000-fce3a12ac2ad admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:24.119035 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876e220-0c77-4e0a-b64c-53fbc0a1cb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:24.119807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876e220-0c77-4e0a-b64c-53fbc0a1cb83 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:24.119993 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876e220-0c77-4e0a-b64c-53fbc0a1cb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:24.120269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876e220-0c77-4e0a-b64c-53fbc0a1cb83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:24.120904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876e220-0c77-4e0a-b64c-53fbc0a1cb83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:24.122818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876e220-0c77-4e0a-b64c-53fbc0a1cb83 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:24.136744 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5d71f4f-55fe-48cb-a169-57945f8e6e98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba748f11c554574b8c084b118700e1d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:24.147430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5d71f4f-55fe-48cb-a169-57945f8e6e98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba748f11c554574b8c084b118700e1d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:24.155418 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1000/2013] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:24 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:29:24.159507 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876e220-0c77-4e0a-b64c-53fbc0a1cb83 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:24.162141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5d71f4f-55fe-48cb-a169-57945f8e6e98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba748f11c554574b8c084b118700e1d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:24.171738 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5d71f4f-55fe-48cb-a169-57945f8e6e98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba748f11c554574b8c084b118700e1d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:24.172511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5d71f4f-55fe-48cb-a169-57945f8e6e98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba748f11c554574b8c084b118700e1d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:24.183586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1014/2014] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:23 2022] DELETE /identity/v3/users/aba748f11c554574b8c084b118700e1d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:24.185129 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04d7e7-7a54-42eb-83a5-7b94959957c0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:24.185647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04d7e7-7a54-42eb-83a5-7b94959957c0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:24.186742 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04d7e7-7a54-42eb-83a5-7b94959957c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:24.206881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-cb04d7e7-7a54-42eb-83a5-7b94959957c0 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:24.208425 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cb04d7e7-7a54-42eb-83a5-7b94959957c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:24.233714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876e220-0c77-4e0a-b64c-53fbc0a1cb83 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:24.279726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1015/2015] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:24.288901 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04d7e7-7a54-42eb-83a5-7b94959957c0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:24.291307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cb04d7e7-7a54-42eb-83a5-7b94959957c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:24.306615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace80a79-bb3c-4287-9c7f-274230633ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:24.307254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace80a79-bb3c-4287-9c7f-274230633ada admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:24.307604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace80a79-bb3c-4287-9c7f-274230633ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:24.307842 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace80a79-bb3c-4287-9c7f-274230633ada admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:24.308531 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace80a79-bb3c-4287-9c7f-274230633ada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:24.312845 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace80a79-bb3c-4287-9c7f-274230633ada admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:24.343737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1001/2016] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:24 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:29:24.345899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace80a79-bb3c-4287-9c7f-274230633ada admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:24.350921 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ace80a79-bb3c-4287-9c7f-274230633ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:24.364311 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04d7e7-7a54-42eb-83a5-7b94959957c0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:24.364506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace80a79-bb3c-4287-9c7f-274230633ada admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:24.368214 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bc763-60db-4138-b529-171fe606abf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:24.368791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bc763-60db-4138-b529-171fe606abf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:24.369070 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bc763-60db-4138-b529-171fe606abf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:24.369188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fc120-a0ca-4c79-a7be-68d1b537f6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:24.369393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bc763-60db-4138-b529-171fe606abf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:24.369745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fc120-a0ca-4c79-a7be-68d1b537f6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:24.370030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fc120-a0ca-4c79-a7be-68d1b537f6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:24.370303 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fc120-a0ca-4c79-a7be-68d1b537f6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:24.370944 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782fc120-a0ca-4c79-a7be-68d1b537f6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:24.372648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782fc120-a0ca-4c79-a7be-68d1b537f6b4 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:24.379989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658bc763-60db-4138-b529-171fe606abf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:24.381624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658bc763-60db-4138-b529-171fe606abf6 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:24.385760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1002/2017] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:29:24.389897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fc120-a0ca-4c79-a7be-68d1b537f6b4 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:24.398702 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1016/2018] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:29:24.407382 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fc120-a0ca-4c79-a7be-68d1b537f6b4 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:24.412187 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf708d5-b989-45d0-a8a2-17711248da87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:24.412815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf708d5-b989-45d0-a8a2-17711248da87 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:24.413092 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf708d5-b989-45d0-a8a2-17711248da87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:24.413365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf708d5-b989-45d0-a8a2-17711248da87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:24.414038 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf708d5-b989-45d0-a8a2-17711248da87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:24.415654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf708d5-b989-45d0-a8a2-17711248da87 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:24.435468 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1003/2019] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:24.437254 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bc763-60db-4138-b529-171fe606abf6 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:24.442366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:24.448811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bc763-60db-4138-b529-171fe606abf6 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:24.455835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c84e0d-7d61-4a9f-bf02-99257cb69716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:24.456580 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c84e0d-7d61-4a9f-bf02-99257cb69716 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:24.456960 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c84e0d-7d61-4a9f-bf02-99257cb69716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:24.457482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c84e0d-7d61-4a9f-bf02-99257cb69716 admin admin] PATH_INFO: `/v3/projects/6fa4238c07c0476095fde3edb52432ef/users/ec0a2c1fdc7945f6ac260c8e8ae7ad4d/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:24.473884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c84e0d-7d61-4a9f-bf02-99257cb69716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fa4238c07c0476095fde3edb52432ef, user_id=ec0a2c1fdc7945f6ac260c8e8ae7ad4d, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:24.476889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c84e0d-7d61-4a9f-bf02-99257cb69716 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:24.490032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1017/2020] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:24 2022] PUT /identity/v3/projects/6fa4238c07c0476095fde3edb52432ef/users/ec0a2c1fdc7945f6ac260c8e8ae7ad4d/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:24.502219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d92ce7-e5e4-4fbd-8ae3-115b7593d4f2 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:24.502567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d92ce7-e5e4-4fbd-8ae3-115b7593d4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:24.502879 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d92ce7-e5e4-4fbd-8ae3-115b7593d4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:24.539758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-46d92ce7-e5e4-4fbd-8ae3-115b7593d4f2 None None] MFA Rules not processed for user `ec0a2c1fdc7945f6ac260c8e8ae7ad4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:24.540258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-46d92ce7-e5e4-4fbd-8ae3-115b7593d4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:24.584452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920deee1-dca8-4355-b67c-43f19a91feda None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:24.584765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920deee1-dca8-4355-b67c-43f19a91feda None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:24.585020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920deee1-dca8-4355-b67c-43f19a91feda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:24.597811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:24.608450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-920deee1-dca8-4355-b67c-43f19a91feda None None] MFA Rules not processed for user `2dcd3dd65d5841049ed20e36e847bb53`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:24.609949 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-920deee1-dca8-4355-b67c-43f19a91feda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:24.651827 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1004/2021] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:24.669365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d92ce7-e5e4-4fbd-8ae3-115b7593d4f2 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:24.671147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-46d92ce7-e5e4-4fbd-8ae3-115b7593d4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:24.692637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1018/2022] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:24.744927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d92ce7-e5e4-4fbd-8ae3-115b7593d4f2 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:24.750577 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fa953-620a-4fe0-81a4-19b68970d27e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:24.751201 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fa953-620a-4fe0-81a4-19b68970d27e service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:24.751492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fa953-620a-4fe0-81a4-19b68970d27e service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:24.751797 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fa953-620a-4fe0-81a4-19b68970d27e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:24.756816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920deee1-dca8-4355-b67c-43f19a91feda None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:24.759730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-920deee1-dca8-4355-b67c-43f19a91feda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:24.766784 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0fa953-620a-4fe0-81a4-19b68970d27e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:24.769549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0fa953-620a-4fe0-81a4-19b68970d27e service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:24.789654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1005/2023] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:29:24.888251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920deee1-dca8-4355-b67c-43f19a91feda None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:24.895112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:24.895816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:24.896100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:24.896370 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] PATH_INFO: `/v3/projects/a85123eaecc84a7dacd2ac22244a0088` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:24.899556 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a85123eaecc84a7dacd2ac22244a0088)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:24.905906 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:25.010421 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a85123eaecc84a7dacd2ac22244a0088', 'request_id': 'req-17531cc7-b908-44c5-ad98-ca1034a03af2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.017624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a85123eaecc84a7dacd2ac22244a0088', 'request_id': 'req-17531cc7-b908-44c5-ad98-ca1034a03af2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.028491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a85123eaecc84a7dacd2ac22244a0088', 'request_id': 'req-17531cc7-b908-44c5-ad98-ca1034a03af2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.041724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] The token cache is being invalidate because project a85123eaecc84a7dacd2ac22244a0088 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:25.042202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.042902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.045225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1019/2024] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:24 2022] DELETE /identity/v3/projects/a85123eaecc84a7dacd2ac22244a0088 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:25.049562 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fa953-620a-4fe0-81a4-19b68970d27e service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:25.062242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fa953-620a-4fe0-81a4-19b68970d27e service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:25.130609 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3ef4a-15ac-4a0e-a09c-7d2de18aa600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:25.131258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3ef4a-15ac-4a0e-a09c-7d2de18aa600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:25.131544 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3ef4a-15ac-4a0e-a09c-7d2de18aa600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:25.131855 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3ef4a-15ac-4a0e-a09c-7d2de18aa600 admin admin] PATH_INFO: `/v3/users/d83716aed9574798bc097b8b737aa4c3` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:25.133015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3ef4a-15ac-4a0e-a09c-7d2de18aa600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d83716aed9574798bc097b8b737aa4c3)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:25.135267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3ef4a-15ac-4a0e-a09c-7d2de18aa600 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:25.200614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-00c3ef4a-15ac-4a0e-a09c-7d2de18aa600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd83716aed9574798bc097b8b737aa4c3'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.217782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-00c3ef4a-15ac-4a0e-a09c-7d2de18aa600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd83716aed9574798bc097b8b737aa4c3'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.227256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-00c3ef4a-15ac-4a0e-a09c-7d2de18aa600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd83716aed9574798bc097b8b737aa4c3'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.228311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-00c3ef4a-15ac-4a0e-a09c-7d2de18aa600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd83716aed9574798bc097b8b737aa4c3'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.238349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-00c3ef4a-15ac-4a0e-a09c-7d2de18aa600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd83716aed9574798bc097b8b737aa4c3'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.240936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:25.250657 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1006/2025] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:25 2022] DELETE /identity/v3/users/d83716aed9574798bc097b8b737aa4c3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:25.594905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:25.596081 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:25.618344 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17531cc7-b908-44c5-ad98-ca1034a03af2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:25.629783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:25.721379 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc761c2f-3a4d-4572-89eb-fab48780a9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:25.722181 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc761c2f-3a4d-4572-89eb-fab48780a9ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:25.722552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc761c2f-3a4d-4572-89eb-fab48780a9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:25.722921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc761c2f-3a4d-4572-89eb-fab48780a9ca admin admin] PATH_INFO: `/v3/projects/e9127c63f3f446e4b43f957ce9801c0d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:25.727612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc761c2f-3a4d-4572-89eb-fab48780a9ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9127c63f3f446e4b43f957ce9801c0d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:25.727612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc761c2f-3a4d-4572-89eb-fab48780a9ca admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:25.783080 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc761c2f-3a4d-4572-89eb-fab48780a9ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9127c63f3f446e4b43f957ce9801c0d', 'request_id': 'req-fc761c2f-3a4d-4572-89eb-fab48780a9ca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.789394 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc761c2f-3a4d-4572-89eb-fab48780a9ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9127c63f3f446e4b43f957ce9801c0d', 'request_id': 'req-fc761c2f-3a4d-4572-89eb-fab48780a9ca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.798505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc761c2f-3a4d-4572-89eb-fab48780a9ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9127c63f3f446e4b43f957ce9801c0d', 'request_id': 'req-fc761c2f-3a4d-4572-89eb-fab48780a9ca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.817133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc761c2f-3a4d-4572-89eb-fab48780a9ca admin admin] The token cache is being invalidate because project e9127c63f3f446e4b43f957ce9801c0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:25.817673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc761c2f-3a4d-4572-89eb-fab48780a9ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.818680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc761c2f-3a4d-4572-89eb-fab48780a9ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:25.820583 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1020/2026] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:25 2022] DELETE /identity/v3/projects/e9127c63f3f446e4b43f957ce9801c0d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:25.825771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d1415-1eaa-4e46-a287-5e1e7c81ff66 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:25.826252 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d1415-1eaa-4e46-a287-5e1e7c81ff66 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:25.826544 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d1415-1eaa-4e46-a287-5e1e7c81ff66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:25.859076 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c51d1415-1eaa-4e46-a287-5e1e7c81ff66 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:25.860512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c51d1415-1eaa-4e46-a287-5e1e7c81ff66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:25.933639 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1007/2027] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:25.942064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0198b6-673a-4f06-bcaa-4f7d4afe3395 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:25.942500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0198b6-673a-4f06-bcaa-4f7d4afe3395 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:25.942788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0198b6-673a-4f06-bcaa-4f7d4afe3395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:25.978685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ee0198b6-673a-4f06-bcaa-4f7d4afe3395 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:25.979928 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ee0198b6-673a-4f06-bcaa-4f7d4afe3395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:25.989198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1021/2028] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:25.993612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d1415-1eaa-4e46-a287-5e1e7c81ff66 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:26.006149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d1415-1eaa-4e46-a287-5e1e7c81ff66 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:26.012566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af60cb9-adc4-4437-aafc-bbe3e03b122c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:26.013191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af60cb9-adc4-4437-aafc-bbe3e03b122c admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:26.014697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af60cb9-adc4-4437-aafc-bbe3e03b122c admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:26.014817 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af60cb9-adc4-4437-aafc-bbe3e03b122c admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:26.016368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af60cb9-adc4-4437-aafc-bbe3e03b122c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:29:26.019912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af60cb9-adc4-4437-aafc-bbe3e03b122c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:26.021565 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af60cb9-adc4-4437-aafc-bbe3e03b122c admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:26.040128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1008/2029] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:29:26.041817 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0198b6-673a-4f06-bcaa-4f7d4afe3395 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:26.052633 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0198b6-673a-4f06-bcaa-4f7d4afe3395 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:26.059528 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2a77d-192b-4f79-838e-e75531e7caed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:26.060253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2a77d-192b-4f79-838e-e75531e7caed admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:26.060530 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2a77d-192b-4f79-838e-e75531e7caed admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:26.061304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2a77d-192b-4f79-838e-e75531e7caed admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:26.061991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2a77d-192b-4f79-838e-e75531e7caed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:26.063936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2a77d-192b-4f79-838e-e75531e7caed admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:26.100077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1022/2030] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:26 2022] POST /identity/v3/projects => generated 407 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:29:26.103969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af60cb9-adc4-4437-aafc-bbe3e03b122c admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:26.118893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af60cb9-adc4-4437-aafc-bbe3e03b122c admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:26.186799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79672bb6-85e7-4aa9-a40a-c20a9acebf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:26.187718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79672bb6-85e7-4aa9-a40a-c20a9acebf42 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:26.188016 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79672bb6-85e7-4aa9-a40a-c20a9acebf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:26.188280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79672bb6-85e7-4aa9-a40a-c20a9acebf42 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:26.188920 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79672bb6-85e7-4aa9-a40a-c20a9acebf42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:26.190860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79672bb6-85e7-4aa9-a40a-c20a9acebf42 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:26.230938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1009/2031] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:26 2022] POST /identity/v3/users => generated 304 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:26.235790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2a77d-192b-4f79-838e-e75531e7caed admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:26.246245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2a77d-192b-4f79-838e-e75531e7caed admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:26.251658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d9e02-be43-44a3-8150-3b1c65b8d1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:26.252269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d9e02-be43-44a3-8150-3b1c65b8d1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:26.252547 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d9e02-be43-44a3-8150-3b1c65b8d1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:26.252828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d9e02-be43-44a3-8150-3b1c65b8d1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:26.253482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d9e02-be43-44a3-8150-3b1c65b8d1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:26.255118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d9e02-be43-44a3-8150-3b1c65b8d1d2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:26.268285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1023/2032] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:26.269430 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79672bb6-85e7-4aa9-a40a-c20a9acebf42 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:26.279660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79672bb6-85e7-4aa9-a40a-c20a9acebf42 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:26.286271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686dc08-a2e4-44ee-bc90-870f3a52e845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:26.286874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686dc08-a2e4-44ee-bc90-870f3a52e845 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:26.287176 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686dc08-a2e4-44ee-bc90-870f3a52e845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:26.287449 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686dc08-a2e4-44ee-bc90-870f3a52e845 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:26.288125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686dc08-a2e4-44ee-bc90-870f3a52e845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:26.289763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686dc08-a2e4-44ee-bc90-870f3a52e845 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:26.302428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1010/2033] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:26.306821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d9e02-be43-44a3-8150-3b1c65b8d1d2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:26.318167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d9e02-be43-44a3-8150-3b1c65b8d1d2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:26.323430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b7ce5-7210-4f90-ae06-7a387a79bea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:26.324203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b7ce5-7210-4f90-ae06-7a387a79bea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:26.324485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b7ce5-7210-4f90-ae06-7a387a79bea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:26.324792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b7ce5-7210-4f90-ae06-7a387a79bea2 admin admin] PATH_INFO: `/v3/projects/696b7f77886b4ca69842e0e1dde12611/users/84f5c29b4ba241b5bf8c6601e6165e1c/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:26.341730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b7ce5-7210-4f90-ae06-7a387a79bea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=696b7f77886b4ca69842e0e1dde12611, user_id=84f5c29b4ba241b5bf8c6601e6165e1c, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:26.344601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b7ce5-7210-4f90-ae06-7a387a79bea2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:26.347416 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09340c-66bb-44a4-9496-aaa9af37b9fb None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:26.348168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09340c-66bb-44a4-9496-aaa9af37b9fb None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:26.348591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09340c-66bb-44a4-9496-aaa9af37b9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:26.358709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1024/2034] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:26 2022] PUT /identity/v3/projects/696b7f77886b4ca69842e0e1dde12611/users/84f5c29b4ba241b5bf8c6601e6165e1c/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:26.369787 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e4431-3d3f-4ef4-97fa-43d3f94acd0b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:26.370075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e4431-3d3f-4ef4-97fa-43d3f94acd0b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:26.370348 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e4431-3d3f-4ef4-97fa-43d3f94acd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:26.370988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-3f09340c-66bb-44a4-9496-aaa9af37b9fb None None] MFA Rules not processed for user `ec0a2c1fdc7945f6ac260c8e8ae7ad4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:26.372279 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3f09340c-66bb-44a4-9496-aaa9af37b9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:26.399525 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-337e4431-3d3f-4ef4-97fa-43d3f94acd0b None None] MFA Rules not processed for user `84f5c29b4ba241b5bf8c6601e6165e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:26.401026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-337e4431-3d3f-4ef4-97fa-43d3f94acd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:26.436527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1011/2035] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:29:26.444869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:26.491512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1025/2036] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:26 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:26.492909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09340c-66bb-44a4-9496-aaa9af37b9fb None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:26.506163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:26.560084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09340c-66bb-44a4-9496-aaa9af37b9fb None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:26.565242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb775a6-1516-4dca-a29a-74d545dbc7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:26.565853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb775a6-1516-4dca-a29a-74d545dbc7df admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:26.566133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb775a6-1516-4dca-a29a-74d545dbc7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:26.566425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb775a6-1516-4dca-a29a-74d545dbc7df admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:26.567060 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb775a6-1516-4dca-a29a-74d545dbc7df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:26.569001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb775a6-1516-4dca-a29a-74d545dbc7df admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:26.600405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1012/2037] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:26 2022] POST /identity/v3/projects => generated 407 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:29:26.604216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e4431-3d3f-4ef4-97fa-43d3f94acd0b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:26.616808 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e4431-3d3f-4ef4-97fa-43d3f94acd0b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:26.702580 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44412fc7-249f-4986-bca7-eb3c18af8035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:26.703498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44412fc7-249f-4986-bca7-eb3c18af8035 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:26.703977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44412fc7-249f-4986-bca7-eb3c18af8035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:26.704563 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44412fc7-249f-4986-bca7-eb3c18af8035 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:26.705328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44412fc7-249f-4986-bca7-eb3c18af8035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:26.708022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44412fc7-249f-4986-bca7-eb3c18af8035 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:26.731228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1026/2038] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:26 2022] POST /identity/v3/users => generated 310 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:29:26.735910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb775a6-1516-4dca-a29a-74d545dbc7df admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:26.747843 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb775a6-1516-4dca-a29a-74d545dbc7df admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:26.754514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1dbc5-16c0-4029-b3c4-1ae113868b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:26.755172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1dbc5-16c0-4029-b3c4-1ae113868b06 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:26.755435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1dbc5-16c0-4029-b3c4-1ae113868b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:26.755747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1dbc5-16c0-4029-b3c4-1ae113868b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:26.756411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1dbc5-16c0-4029-b3c4-1ae113868b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:26.758070 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1dbc5-16c0-4029-b3c4-1ae113868b06 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:26.770243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1013/2039] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:26.774491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44412fc7-249f-4986-bca7-eb3c18af8035 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:26.786200 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44412fc7-249f-4986-bca7-eb3c18af8035 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:26.791346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789c427-2f7c-4583-a5cc-c6e480baedd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:26.792220 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789c427-2f7c-4583-a5cc-c6e480baedd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:26.792598 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789c427-2f7c-4583-a5cc-c6e480baedd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:26.792973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789c427-2f7c-4583-a5cc-c6e480baedd1 admin admin] PATH_INFO: `/v3/projects/ed2ebd80d3104853afea45a16496df14/users/53a8a3bfdaa644218ed405b066c7372a/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:26.811556 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789c427-2f7c-4583-a5cc-c6e480baedd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2ebd80d3104853afea45a16496df14, user_id=53a8a3bfdaa644218ed405b066c7372a, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:26.814512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789c427-2f7c-4583-a5cc-c6e480baedd1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:26.838316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1dbc5-16c0-4029-b3c4-1ae113868b06 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:26.839623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1027/2040] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:26 2022] PUT /identity/v3/projects/ed2ebd80d3104853afea45a16496df14/users/53a8a3bfdaa644218ed405b066c7372a/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:26.853439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1dbc5-16c0-4029-b3c4-1ae113868b06 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:26.967046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99e8f7-a71f-4903-a0af-737023853b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:26.967046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99e8f7-a71f-4903-a0af-737023853b34 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:26.967046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99e8f7-a71f-4903-a0af-737023853b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:26.967046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99e8f7-a71f-4903-a0af-737023853b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:26.967957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99e8f7-a71f-4903-a0af-737023853b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:26.969587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99e8f7-a71f-4903-a0af-737023853b34 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:26.992126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1014/2041] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:26.996348 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789c427-2f7c-4583-a5cc-c6e480baedd1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:27.029057 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789c427-2f7c-4583-a5cc-c6e480baedd1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:27.036400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee23144-5b29-4a8a-9c78-4294f8d58ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:27.037080 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee23144-5b29-4a8a-9c78-4294f8d58ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:27.037368 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee23144-5b29-4a8a-9c78-4294f8d58ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:27.037630 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee23144-5b29-4a8a-9c78-4294f8d58ce3 admin admin] PATH_INFO: `/v3/projects/ed2ebd80d3104853afea45a16496df14/users/53a8a3bfdaa644218ed405b066c7372a/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:27.039685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee23144-5b29-4a8a-9c78-4294f8d58ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2ebd80d3104853afea45a16496df14, user_id=53a8a3bfdaa644218ed405b066c7372a, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:27.042565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee23144-5b29-4a8a-9c78-4294f8d58ce3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:27.069429 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1028/2042] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:26 2022] PUT /identity/v3/projects/ed2ebd80d3104853afea45a16496df14/users/53a8a3bfdaa644218ed405b066c7372a/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:27.079968 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910748-2d72-4a1c-9755-2c58d395cc14 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:27.080625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910748-2d72-4a1c-9755-2c58d395cc14 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:27.081107 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910748-2d72-4a1c-9755-2c58d395cc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:27.140729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-76910748-2d72-4a1c-9755-2c58d395cc14 None None] MFA Rules not processed for user `53a8a3bfdaa644218ed405b066c7372a`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:27.142242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-76910748-2d72-4a1c-9755-2c58d395cc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:27.250177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1015/2043] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:27 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:29:27.261892 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee23144-5b29-4a8a-9c78-4294f8d58ce3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:27.264045 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7ee23144-5b29-4a8a-9c78-4294f8d58ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:27.344413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee23144-5b29-4a8a-9c78-4294f8d58ce3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:27.351249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf6fc0-ccf2-4a3d-a435-410ae20411f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:27.352411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf6fc0-ccf2-4a3d-a435-410ae20411f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:27.352848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf6fc0-ccf2-4a3d-a435-410ae20411f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:27.353280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf6fc0-ccf2-4a3d-a435-410ae20411f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:27.388831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf6fc0-ccf2-4a3d-a435-410ae20411f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:27.391316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf6fc0-ccf2-4a3d-a435-410ae20411f7 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:27.416945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1029/2044] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:27 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:27.645849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76910748-2d72-4a1c-9755-2c58d395cc14 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:27.656955 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76910748-2d72-4a1c-9755-2c58d395cc14 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:27.660707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ab0b6-c9b4-47c1-9c56-58bd96297a46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:27.661424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ab0b6-c9b4-47c1-9c56-58bd96297a46 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:27.661777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ab0b6-c9b4-47c1-9c56-58bd96297a46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:27.662122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ab0b6-c9b4-47c1-9c56-58bd96297a46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:27.666936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:27.765698 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ab0b6-c9b4-47c1-9c56-58bd96297a46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:27.767226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ab0b6-c9b4-47c1-9c56-58bd96297a46 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:27.778729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1016/2045] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:27 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:32.394164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf6fc0-ccf2-4a3d-a435-410ae20411f7 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:32.396535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9bdf6fc0-ccf2-4a3d-a435-410ae20411f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:32.430794 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:32.476859 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf6fc0-ccf2-4a3d-a435-410ae20411f7 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:32.486261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4164b1-1979-42e1-b78c-fa9a757b1b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:32.487339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4164b1-1979-42e1-b78c-fa9a757b1b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:32.487872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4164b1-1979-42e1-b78c-fa9a757b1b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:32.488351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4164b1-1979-42e1-b78c-fa9a757b1b09 admin admin] PATH_INFO: `/v3/users/98da6780db2d43d5be8e971dd2d86119` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:32.489998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4164b1-1979-42e1-b78c-fa9a757b1b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98da6780db2d43d5be8e971dd2d86119)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:32.492347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4164b1-1979-42e1-b78c-fa9a757b1b09 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:32.577832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d4164b1-1979-42e1-b78c-fa9a757b1b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98da6780db2d43d5be8e971dd2d86119'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:32.586894 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d4164b1-1979-42e1-b78c-fa9a757b1b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98da6780db2d43d5be8e971dd2d86119'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:32.605863 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d4164b1-1979-42e1-b78c-fa9a757b1b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98da6780db2d43d5be8e971dd2d86119'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:32.620175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d4164b1-1979-42e1-b78c-fa9a757b1b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98da6780db2d43d5be8e971dd2d86119'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:32.620801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6d4164b1-1979-42e1-b78c-fa9a757b1b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98da6780db2d43d5be8e971dd2d86119'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:32.627987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1030/2046] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:32 2022] DELETE /identity/v3/users/98da6780db2d43d5be8e971dd2d86119 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:32.812896 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ab0b6-c9b4-47c1-9c56-58bd96297a46 service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:32.827362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ab0b6-c9b4-47c1-9c56-58bd96297a46 service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:32.913766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:32.914421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:32.914709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:32.914982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8 admin admin] PATH_INFO: `/v3/projects/811d8512813f49d79facec817fc5d03e` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:32.916164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=811d8512813f49d79facec817fc5d03e)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:32.918117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:32.978542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '811d8512813f49d79facec817fc5d03e', 'request_id': 'req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:32.984629 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '811d8512813f49d79facec817fc5d03e', 'request_id': 'req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:32.994289 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '811d8512813f49d79facec817fc5d03e', 'request_id': 'req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:33.009874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8 admin admin] The token cache is being invalidate because project 811d8512813f49d79facec817fc5d03e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:33.010151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:33.010984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f532a4ae-c079-44a1-b3c3-bc3ec66affd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:33.012821 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1017/2047] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:32 2022] DELETE /identity/v3/projects/811d8512813f49d79facec817fc5d03e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:33.018253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f31717-611a-4c02-a405-b7aa210761f3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:33.018544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f31717-611a-4c02-a405-b7aa210761f3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:33.018822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f31717-611a-4c02-a405-b7aa210761f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:33.043350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-65f31717-611a-4c02-a405-b7aa210761f3 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:33.045022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-65f31717-611a-4c02-a405-b7aa210761f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:33.133094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1031/2048] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:33.141948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e613f23-ee2f-4c21-aef4-eee752a602b3 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:33.142235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e613f23-ee2f-4c21-aef4-eee752a602b3 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:33.142518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e613f23-ee2f-4c21-aef4-eee752a602b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:33.164529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-6e613f23-ee2f-4c21-aef4-eee752a602b3 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:33.165768 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6e613f23-ee2f-4c21-aef4-eee752a602b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:33.179598 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1018/2049] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:33.184991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f31717-611a-4c02-a405-b7aa210761f3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:33.195444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f31717-611a-4c02-a405-b7aa210761f3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:33.200848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9fe3f-6b64-4b4f-a9e5-0ed5b0a67565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:33.202235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9fe3f-6b64-4b4f-a9e5-0ed5b0a67565 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:33.202617 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9fe3f-6b64-4b4f-a9e5-0ed5b0a67565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:33.202992 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9fe3f-6b64-4b4f-a9e5-0ed5b0a67565 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:33.203802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9fe3f-6b64-4b4f-a9e5-0ed5b0a67565 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:29:33.204226 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9fe3f-6b64-4b4f-a9e5-0ed5b0a67565 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:33.205991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9fe3f-6b64-4b4f-a9e5-0ed5b0a67565 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:33.224347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1032/2050] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:29:33.225131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e613f23-ee2f-4c21-aef4-eee752a602b3 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:33.235644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e613f23-ee2f-4c21-aef4-eee752a602b3 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:33.240511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbf22e-3e0a-489a-b350-7a392850795a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:33.241141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbf22e-3e0a-489a-b350-7a392850795a admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:33.241567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbf22e-3e0a-489a-b350-7a392850795a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:33.241846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbf22e-3e0a-489a-b350-7a392850795a admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:33.242495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbf22e-3e0a-489a-b350-7a392850795a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:33.244520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbf22e-3e0a-489a-b350-7a392850795a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:33.283605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1019/2051] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:33 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:33.301053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9fe3f-6b64-4b4f-a9e5-0ed5b0a67565 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:33.311218 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9fe3f-6b64-4b4f-a9e5-0ed5b0a67565 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:33.351190 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:33.385299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9fba8-6553-43bf-b8a9-0e08e8a8d6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:33.385923 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9fba8-6553-43bf-b8a9-0e08e8a8d6ce admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:33.386205 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9fba8-6553-43bf-b8a9-0e08e8a8d6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:33.386483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9fba8-6553-43bf-b8a9-0e08e8a8d6ce admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:33.387127 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9fba8-6553-43bf-b8a9-0e08e8a8d6ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:33.389705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9fba8-6553-43bf-b8a9-0e08e8a8d6ce admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:33.423873 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1033/2052] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:33 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:33.426473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbf22e-3e0a-489a-b350-7a392850795a admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:33.441992 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbf22e-3e0a-489a-b350-7a392850795a admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:33.447488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f104b-a314-4dda-b9e8-1ad4515cccc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:33.448158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f104b-a314-4dda-b9e8-1ad4515cccc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:33.448446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f104b-a314-4dda-b9e8-1ad4515cccc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:33.448726 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f104b-a314-4dda-b9e8-1ad4515cccc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:33.449373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f104b-a314-4dda-b9e8-1ad4515cccc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:33.451556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f104b-a314-4dda-b9e8-1ad4515cccc6 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:33.453167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:33.464982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1020/2053] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:33.469922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9fba8-6553-43bf-b8a9-0e08e8a8d6ce admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:33.495110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9fba8-6553-43bf-b8a9-0e08e8a8d6ce admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:33.500595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2e92e-b7ef-4906-a466-6871b7a93229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:33.501229 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2e92e-b7ef-4906-a466-6871b7a93229 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:33.501508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2e92e-b7ef-4906-a466-6871b7a93229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:33.501778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2e92e-b7ef-4906-a466-6871b7a93229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:33.502417 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2e92e-b7ef-4906-a466-6871b7a93229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:33.504086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2e92e-b7ef-4906-a466-6871b7a93229 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:33.516201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1034/2054] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:33.521081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f104b-a314-4dda-b9e8-1ad4515cccc6 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:33.533000 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f104b-a314-4dda-b9e8-1ad4515cccc6 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:33.538637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feff7f0-9c06-480f-8fce-1ca00b53e0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:33.539405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feff7f0-9c06-480f-8fce-1ca00b53e0e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:33.539881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feff7f0-9c06-480f-8fce-1ca00b53e0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:33.540236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feff7f0-9c06-480f-8fce-1ca00b53e0e7 admin admin] PATH_INFO: `/v3/projects/3fb450976bcd4590bf2457e1d6fbbd10/users/c626ba7e4b954484ad8bdf23ce5fe87b/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:33.566706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feff7f0-9c06-480f-8fce-1ca00b53e0e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fb450976bcd4590bf2457e1d6fbbd10, user_id=c626ba7e4b954484ad8bdf23ce5fe87b, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:33.569983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feff7f0-9c06-480f-8fce-1ca00b53e0e7 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:33.606197 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1021/2055] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:33 2022] PUT /identity/v3/projects/3fb450976bcd4590bf2457e1d6fbbd10/users/c626ba7e4b954484ad8bdf23ce5fe87b/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:33.614616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a56758-ee79-410e-bbab-1ae2abd56e27 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:33.615214 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a56758-ee79-410e-bbab-1ae2abd56e27 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:33.615393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a56758-ee79-410e-bbab-1ae2abd56e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:33.673539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-22a56758-ee79-410e-bbab-1ae2abd56e27 None None] MFA Rules not processed for user `c626ba7e4b954484ad8bdf23ce5fe87b`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:33.675392 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-22a56758-ee79-410e-bbab-1ae2abd56e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:33.782940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1035/2056] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:33.789783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feff7f0-9c06-480f-8fce-1ca00b53e0e7 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:33.792100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6feff7f0-9c06-480f-8fce-1ca00b53e0e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:33.898522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feff7f0-9c06-480f-8fce-1ca00b53e0e7 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:33.906299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7c443-627e-44e6-82d2-cdd9fbf86ac9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:33.907644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7c443-627e-44e6-82d2-cdd9fbf86ac9 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:33.907950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7c443-627e-44e6-82d2-cdd9fbf86ac9 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:33.908228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7c443-627e-44e6-82d2-cdd9fbf86ac9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:33.917929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7c443-627e-44e6-82d2-cdd9fbf86ac9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:33.919187 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7c443-627e-44e6-82d2-cdd9fbf86ac9 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:33.931433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1022/2057] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:34.237449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a56758-ee79-410e-bbab-1ae2abd56e27 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:34.334109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a56758-ee79-410e-bbab-1ae2abd56e27 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:34.338913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bd4e4-326c-44ee-9598-f0f77d2b1dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:34.339554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bd4e4-326c-44ee-9598-f0f77d2b1dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:34.339865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bd4e4-326c-44ee-9598-f0f77d2b1dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:34.340141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bd4e4-326c-44ee-9598-f0f77d2b1dc5 admin admin] PATH_INFO: `/v3/users/c626ba7e4b954484ad8bdf23ce5fe87b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:34.341304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bd4e4-326c-44ee-9598-f0f77d2b1dc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c626ba7e4b954484ad8bdf23ce5fe87b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:34.343868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bd4e4-326c-44ee-9598-f0f77d2b1dc5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:34.386630 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:34.393642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-256bd4e4-326c-44ee-9598-f0f77d2b1dc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c626ba7e4b954484ad8bdf23ce5fe87b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:34.401282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-256bd4e4-326c-44ee-9598-f0f77d2b1dc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c626ba7e4b954484ad8bdf23ce5fe87b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:34.414164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-256bd4e4-326c-44ee-9598-f0f77d2b1dc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c626ba7e4b954484ad8bdf23ce5fe87b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:34.422552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-256bd4e4-326c-44ee-9598-f0f77d2b1dc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c626ba7e4b954484ad8bdf23ce5fe87b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:34.423268 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-256bd4e4-326c-44ee-9598-f0f77d2b1dc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c626ba7e4b954484ad8bdf23ce5fe87b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:34.429907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1036/2058] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:34 2022] DELETE /identity/v3/users/c626ba7e4b954484ad8bdf23ce5fe87b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:34.434602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270949a-137f-4e0c-b29a-6a4d28673b35 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:34.434891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270949a-137f-4e0c-b29a-6a4d28673b35 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:34.435166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270949a-137f-4e0c-b29a-6a4d28673b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:34.455683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-5270949a-137f-4e0c-b29a-6a4d28673b35 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:34.457073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5270949a-137f-4e0c-b29a-6a4d28673b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:34.518455 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1023/2059] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:34.528321 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bd4e4-326c-44ee-9598-f0f77d2b1dc5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:34.530569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-256bd4e4-326c-44ee-9598-f0f77d2b1dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:34.591020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bd4e4-326c-44ee-9598-f0f77d2b1dc5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:34.595778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0f933-c3a2-445a-a5bb-376d532c22c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:34.596475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0f933-c3a2-445a-a5bb-376d532c22c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:34.596867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0f933-c3a2-445a-a5bb-376d532c22c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:34.597267 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0f933-c3a2-445a-a5bb-376d532c22c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:34.610165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0f933-c3a2-445a-a5bb-376d532c22c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:34.611718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0f933-c3a2-445a-a5bb-376d532c22c7 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:34.621558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1037/2060] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:34.898826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270949a-137f-4e0c-b29a-6a4d28673b35 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:34.900502 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5270949a-137f-4e0c-b29a-6a4d28673b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:34.916878 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270949a-137f-4e0c-b29a-6a4d28673b35 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:34.922575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ccaf1-4435-430c-9fcb-07d41e61f134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:34.923235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ccaf1-4435-430c-9fcb-07d41e61f134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:34.923536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ccaf1-4435-430c-9fcb-07d41e61f134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:34.923814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ccaf1-4435-430c-9fcb-07d41e61f134 admin admin] PATH_INFO: `/v3/projects/3fb450976bcd4590bf2457e1d6fbbd10` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:34.924980 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ccaf1-4435-430c-9fcb-07d41e61f134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fb450976bcd4590bf2457e1d6fbbd10)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:34.926978 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ccaf1-4435-430c-9fcb-07d41e61f134 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:34.976256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-497ccaf1-4435-430c-9fcb-07d41e61f134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fb450976bcd4590bf2457e1d6fbbd10', 'request_id': 'req-497ccaf1-4435-430c-9fcb-07d41e61f134'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:34.982028 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:34.989122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-497ccaf1-4435-430c-9fcb-07d41e61f134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fb450976bcd4590bf2457e1d6fbbd10', 'request_id': 'req-497ccaf1-4435-430c-9fcb-07d41e61f134'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:34.999660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-497ccaf1-4435-430c-9fcb-07d41e61f134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fb450976bcd4590bf2457e1d6fbbd10', 'request_id': 'req-497ccaf1-4435-430c-9fcb-07d41e61f134'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:35.011937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-497ccaf1-4435-430c-9fcb-07d41e61f134 admin admin] The token cache is being invalidate because project 3fb450976bcd4590bf2457e1d6fbbd10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:35.012230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-497ccaf1-4435-430c-9fcb-07d41e61f134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:35.013114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-497ccaf1-4435-430c-9fcb-07d41e61f134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:35.014618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1024/2061] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:34 2022] DELETE /identity/v3/projects/3fb450976bcd4590bf2457e1d6fbbd10 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:35.019750 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7a362-b056-46d5-90ed-e2f759da1232 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:35.020059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7a362-b056-46d5-90ed-e2f759da1232 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:35.020335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7a362-b056-46d5-90ed-e2f759da1232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:35.040672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c5f7a362-b056-46d5-90ed-e2f759da1232 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:35.041905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c5f7a362-b056-46d5-90ed-e2f759da1232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:35.109719 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1038/2062] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:35.119916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cc475-0a01-48d4-8acd-3f5bbd75f48d None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:35.120209 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cc475-0a01-48d4-8acd-3f5bbd75f48d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:35.120489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cc475-0a01-48d4-8acd-3f5bbd75f48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:35.145366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-423cc475-0a01-48d4-8acd-3f5bbd75f48d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:35.145366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-423cc475-0a01-48d4-8acd-3f5bbd75f48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:35.162914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1025/2063] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:35.166943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7a362-b056-46d5-90ed-e2f759da1232 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:35.177224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7a362-b056-46d5-90ed-e2f759da1232 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:35.182376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c444a19-2ba2-4f91-9e60-f3c3eba9a0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:35.182979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c444a19-2ba2-4f91-9e60-f3c3eba9a0ae admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:35.183265 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c444a19-2ba2-4f91-9e60-f3c3eba9a0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:35.183536 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c444a19-2ba2-4f91-9e60-f3c3eba9a0ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:35.184219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c444a19-2ba2-4f91-9e60-f3c3eba9a0ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:29:35.184522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c444a19-2ba2-4f91-9e60-f3c3eba9a0ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:35.186687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c444a19-2ba2-4f91-9e60-f3c3eba9a0ae admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:35.202001 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1039/2064] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:29:35.203163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cc475-0a01-48d4-8acd-3f5bbd75f48d None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:35.212451 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cc475-0a01-48d4-8acd-3f5bbd75f48d None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:35.217090 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c1f18-78ec-43bd-93d0-bc7ffc001ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:35.217681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c1f18-78ec-43bd-93d0-bc7ffc001ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:35.217963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c1f18-78ec-43bd-93d0-bc7ffc001ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:35.219181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c1f18-78ec-43bd-93d0-bc7ffc001ee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:35.219845 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c1f18-78ec-43bd-93d0-bc7ffc001ee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:35.221758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c1f18-78ec-43bd-93d0-bc7ffc001ee9 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:35.253141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1026/2065] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:35 2022] POST /identity/v3/projects => generated 355 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:35.254259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c444a19-2ba2-4f91-9e60-f3c3eba9a0ae admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:35.264485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c444a19-2ba2-4f91-9e60-f3c3eba9a0ae admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:35.310188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:35.333146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421afe3-6120-4560-81fd-d1ddcd39bef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:35.333872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421afe3-6120-4560-81fd-d1ddcd39bef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:35.334162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421afe3-6120-4560-81fd-d1ddcd39bef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:35.334439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421afe3-6120-4560-81fd-d1ddcd39bef0 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:35.335082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a421afe3-6120-4560-81fd-d1ddcd39bef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:35.337090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a421afe3-6120-4560-81fd-d1ddcd39bef0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:35.365178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1040/2066] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:35 2022] POST /identity/v3/users => generated 278 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:35.366058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c1f18-78ec-43bd-93d0-bc7ffc001ee9 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:35.377269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c1f18-78ec-43bd-93d0-bc7ffc001ee9 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:35.382284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5ecfd-3905-496a-8370-ef69f6c8cff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:35.382900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5ecfd-3905-496a-8370-ef69f6c8cff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:35.383180 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5ecfd-3905-496a-8370-ef69f6c8cff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:35.384705 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5ecfd-3905-496a-8370-ef69f6c8cff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:35.384705 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5ecfd-3905-496a-8370-ef69f6c8cff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:35.385735 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5ecfd-3905-496a-8370-ef69f6c8cff9 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:35.402616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1027/2067] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:35.404104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421afe3-6120-4560-81fd-d1ddcd39bef0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:35.420811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421afe3-6120-4560-81fd-d1ddcd39bef0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:35.426837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa36dc4-d534-42f5-890d-771df9c37386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:35.428332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa36dc4-d534-42f5-890d-771df9c37386 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:35.428658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa36dc4-d534-42f5-890d-771df9c37386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:35.428944 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa36dc4-d534-42f5-890d-771df9c37386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:35.430099 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa36dc4-d534-42f5-890d-771df9c37386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:35.431614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa36dc4-d534-42f5-890d-771df9c37386 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:35.450336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1041/2068] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:29:35.462110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5ecfd-3905-496a-8370-ef69f6c8cff9 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:35.473065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5ecfd-3905-496a-8370-ef69f6c8cff9 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:35.480715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc33d8-9791-46ae-8ff5-d9282a16469b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:35.481386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc33d8-9791-46ae-8ff5-d9282a16469b admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:35.481711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc33d8-9791-46ae-8ff5-d9282a16469b admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:35.482841 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc33d8-9791-46ae-8ff5-d9282a16469b admin admin] PATH_INFO: `/v3/projects/a54f57b551e8417490cd01e11967db9d/users/638ad311373f44edbccb5b6c3fa971c0/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:35.498591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc33d8-9791-46ae-8ff5-d9282a16469b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a54f57b551e8417490cd01e11967db9d, user_id=638ad311373f44edbccb5b6c3fa971c0, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:35.501372 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc33d8-9791-46ae-8ff5-d9282a16469b admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:35.516151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1028/2069] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:35 2022] PUT /identity/v3/projects/a54f57b551e8417490cd01e11967db9d/users/638ad311373f44edbccb5b6c3fa971c0/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:35.526569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a22e72-5e9a-42bd-8652-495d247163de None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:35.526857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a22e72-5e9a-42bd-8652-495d247163de None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:35.527172 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a22e72-5e9a-42bd-8652-495d247163de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:35.557357 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-39a22e72-5e9a-42bd-8652-495d247163de None None] MFA Rules not processed for user `638ad311373f44edbccb5b6c3fa971c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:35.559414 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-39a22e72-5e9a-42bd-8652-495d247163de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:35.640639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1042/2070] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:35 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:35.656513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc33d8-9791-46ae-8ff5-d9282a16469b admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:35.745023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc33d8-9791-46ae-8ff5-d9282a16469b admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:35.750590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb33814-478d-40fd-840a-a55ffc94cfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:35.751321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb33814-478d-40fd-840a-a55ffc94cfc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:35.751736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb33814-478d-40fd-840a-a55ffc94cfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:35.752112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb33814-478d-40fd-840a-a55ffc94cfc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:35.752975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb33814-478d-40fd-840a-a55ffc94cfc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:35.755379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb33814-478d-40fd-840a-a55ffc94cfc0 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:35.790399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1029/2071] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:35 2022] POST /identity/v3/projects => generated 353 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:29:35.796516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a22e72-5e9a-42bd-8652-495d247163de None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:35.811908 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a22e72-5e9a-42bd-8652-495d247163de None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:35.874231 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb15f7-c2b5-4f84-953a-6e8cb54546e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:35.874848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb15f7-c2b5-4f84-953a-6e8cb54546e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:35.875130 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb15f7-c2b5-4f84-953a-6e8cb54546e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:35.875422 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb15f7-c2b5-4f84-953a-6e8cb54546e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:35.876107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb15f7-c2b5-4f84-953a-6e8cb54546e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:35.878026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb15f7-c2b5-4f84-953a-6e8cb54546e7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:35.899299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1043/2072] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:35 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:35.903430 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb33814-478d-40fd-840a-a55ffc94cfc0 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:35.915119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb33814-478d-40fd-840a-a55ffc94cfc0 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:35.920507 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807d8de-b33a-47ea-8c6b-d7a32f201c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:35.921173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807d8de-b33a-47ea-8c6b-d7a32f201c8d admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:35.921461 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807d8de-b33a-47ea-8c6b-d7a32f201c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:35.921738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807d8de-b33a-47ea-8c6b-d7a32f201c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:35.922437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807d8de-b33a-47ea-8c6b-d7a32f201c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:35.924081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807d8de-b33a-47ea-8c6b-d7a32f201c8d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:35.936082 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1030/2073] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:35.940290 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb15f7-c2b5-4f84-953a-6e8cb54546e7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:35.950673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb15f7-c2b5-4f84-953a-6e8cb54546e7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:35.956647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e0985-df15-4db1-80aa-2628765832d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:35.957341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e0985-df15-4db1-80aa-2628765832d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:35.957619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e0985-df15-4db1-80aa-2628765832d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:35.957894 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e0985-df15-4db1-80aa-2628765832d2 admin admin] PATH_INFO: `/v3/projects/8dd90954a4984344b7b11250d749b357/users/62a9ea84ff20455996860080dd31d3d0/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:35.974292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e0985-df15-4db1-80aa-2628765832d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd90954a4984344b7b11250d749b357, user_id=62a9ea84ff20455996860080dd31d3d0, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:35.977001 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e0985-df15-4db1-80aa-2628765832d2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:35.990693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1044/2074] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:35 2022] PUT /identity/v3/projects/8dd90954a4984344b7b11250d749b357/users/62a9ea84ff20455996860080dd31d3d0/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:35.994714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807d8de-b33a-47ea-8c6b-d7a32f201c8d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:36.004820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807d8de-b33a-47ea-8c6b-d7a32f201c8d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:36.055322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:36.067958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5494ea3-39ca-410a-ba28-549396f84a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:36.068644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5494ea3-39ca-410a-ba28-549396f84a36 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:36.068926 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5494ea3-39ca-410a-ba28-549396f84a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:36.069236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5494ea3-39ca-410a-ba28-549396f84a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:36.069877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5494ea3-39ca-410a-ba28-549396f84a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:36.071554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5494ea3-39ca-410a-ba28-549396f84a36 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:36.082048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1031/2075] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:36.086234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e0985-df15-4db1-80aa-2628765832d2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:36.097030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e0985-df15-4db1-80aa-2628765832d2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:36.101541 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df72e23-bc17-48c2-9ec5-53e45c58a222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:36.102117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df72e23-bc17-48c2-9ec5-53e45c58a222 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:36.102396 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df72e23-bc17-48c2-9ec5-53e45c58a222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:36.102668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df72e23-bc17-48c2-9ec5-53e45c58a222 admin admin] PATH_INFO: `/v3/projects/8dd90954a4984344b7b11250d749b357/users/62a9ea84ff20455996860080dd31d3d0/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:36.104569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df72e23-bc17-48c2-9ec5-53e45c58a222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd90954a4984344b7b11250d749b357, user_id=62a9ea84ff20455996860080dd31d3d0, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:36.107166 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df72e23-bc17-48c2-9ec5-53e45c58a222 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:36.117164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1045/2076] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:36 2022] PUT /identity/v3/projects/8dd90954a4984344b7b11250d749b357/users/62a9ea84ff20455996860080dd31d3d0/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:36.127806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf448e0-ae11-43f4-bfbc-b4ed7c855b86 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:36.128102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf448e0-ae11-43f4-bfbc-b4ed7c855b86 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:36.128382 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf448e0-ae11-43f4-bfbc-b4ed7c855b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:36.159291 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-faf448e0-ae11-43f4-bfbc-b4ed7c855b86 None None] MFA Rules not processed for user `62a9ea84ff20455996860080dd31d3d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:36.160543 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-faf448e0-ae11-43f4-bfbc-b4ed7c855b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:36.229265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1032/2077] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:36 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:36.242140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df72e23-bc17-48c2-9ec5-53e45c58a222 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:36.244299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0df72e23-bc17-48c2-9ec5-53e45c58a222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:36.254808 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:36.304841 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df72e23-bc17-48c2-9ec5-53e45c58a222 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:36.308650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f60e8d-0f42-45ba-837d-62bd8cfc6ee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:36.309259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f60e8d-0f42-45ba-837d-62bd8cfc6ee8 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:36.309540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f60e8d-0f42-45ba-837d-62bd8cfc6ee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:36.309812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f60e8d-0f42-45ba-837d-62bd8cfc6ee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:36.319595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f60e8d-0f42-45ba-837d-62bd8cfc6ee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:36.324653 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f60e8d-0f42-45ba-837d-62bd8cfc6ee8 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:36.330719 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1046/2078] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:36 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:36.412029 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf448e0-ae11-43f4-bfbc-b4ed7c855b86 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:36.421813 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf448e0-ae11-43f4-bfbc-b4ed7c855b86 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:36.425360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d084cc-c74a-403f-b5e2-1b59b3f3679a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:36.425948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d084cc-c74a-403f-b5e2-1b59b3f3679a service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:36.426224 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d084cc-c74a-403f-b5e2-1b59b3f3679a service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:36.426493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d084cc-c74a-403f-b5e2-1b59b3f3679a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:36.484136 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d084cc-c74a-403f-b5e2-1b59b3f3679a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:36.485659 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d084cc-c74a-403f-b5e2-1b59b3f3679a service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:36.495009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1033/2079] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:36.649993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f60e8d-0f42-45ba-837d-62bd8cfc6ee8 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:36.651693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-71f60e8d-0f42-45ba-837d-62bd8cfc6ee8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:36.722748 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f60e8d-0f42-45ba-837d-62bd8cfc6ee8 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:36.727300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201f331-ae4f-4121-a920-f5357ddceb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:36.727984 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201f331-ae4f-4121-a920-f5357ddceb20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:36.728263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201f331-ae4f-4121-a920-f5357ddceb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:36.728540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201f331-ae4f-4121-a920-f5357ddceb20 admin admin] PATH_INFO: `/v3/users/84f5c29b4ba241b5bf8c6601e6165e1c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:36.729658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201f331-ae4f-4121-a920-f5357ddceb20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f5c29b4ba241b5bf8c6601e6165e1c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:36.731620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201f331-ae4f-4121-a920-f5357ddceb20 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:36.788784 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4201f331-ae4f-4121-a920-f5357ddceb20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f5c29b4ba241b5bf8c6601e6165e1c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:36.800107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4201f331-ae4f-4121-a920-f5357ddceb20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f5c29b4ba241b5bf8c6601e6165e1c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:36.820486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4201f331-ae4f-4121-a920-f5357ddceb20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f5c29b4ba241b5bf8c6601e6165e1c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:36.835794 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4201f331-ae4f-4121-a920-f5357ddceb20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f5c29b4ba241b5bf8c6601e6165e1c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:36.836774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4201f331-ae4f-4121-a920-f5357ddceb20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f5c29b4ba241b5bf8c6601e6165e1c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:36.847626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1047/2080] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:36 2022] DELETE /identity/v3/users/84f5c29b4ba241b5bf8c6601e6165e1c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:36.856863 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fe6f4-d7ab-4e15-901c-06a243eba364 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:36.857630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fe6f4-d7ab-4e15-901c-06a243eba364 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:36.857909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fe6f4-d7ab-4e15-901c-06a243eba364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:36.883621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-636fe6f4-d7ab-4e15-901c-06a243eba364 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:36.884836 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-636fe6f4-d7ab-4e15-901c-06a243eba364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:36.965187 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1034/2081] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:36.977902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201f331-ae4f-4121-a920-f5357ddceb20 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:36.979831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4201f331-ae4f-4121-a920-f5357ddceb20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:37.025519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:37.056357 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201f331-ae4f-4121-a920-f5357ddceb20 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:37.060012 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ce14d-3ad6-4b80-b072-674f9b4fb673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:37.061402 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ce14d-3ad6-4b80-b072-674f9b4fb673 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:37.061683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ce14d-3ad6-4b80-b072-674f9b4fb673 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:37.061954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ce14d-3ad6-4b80-b072-674f9b4fb673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:37.071768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ce14d-3ad6-4b80-b072-674f9b4fb673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:37.073184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ce14d-3ad6-4b80-b072-674f9b4fb673 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:37.083318 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1048/2082] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:37.342114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fe6f4-d7ab-4e15-901c-06a243eba364 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:37.343777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-636fe6f4-d7ab-4e15-901c-06a243eba364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:37.359381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fe6f4-d7ab-4e15-901c-06a243eba364 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:37.369278 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:37.370324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:37.370614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:37.370881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] PATH_INFO: `/v3/projects/696b7f77886b4ca69842e0e1dde12611` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:37.372080 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=696b7f77886b4ca69842e0e1dde12611)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:37.374525 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:37.383897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:37.440440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '696b7f77886b4ca69842e0e1dde12611', 'request_id': 'req-91f78d4a-83e1-4e37-8687-7dda852e6ace'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:37.455381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '696b7f77886b4ca69842e0e1dde12611', 'request_id': 'req-91f78d4a-83e1-4e37-8687-7dda852e6ace'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:37.465717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '696b7f77886b4ca69842e0e1dde12611', 'request_id': 'req-91f78d4a-83e1-4e37-8687-7dda852e6ace'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:37.481072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] The token cache is being invalidate because project 696b7f77886b4ca69842e0e1dde12611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:37.481379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:37.483061 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:37.484860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1035/2083] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:37 2022] DELETE /identity/v3/projects/696b7f77886b4ca69842e0e1dde12611 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:37.488862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ce14d-3ad6-4b80-b072-674f9b4fb673 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:37.500133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ce14d-3ad6-4b80-b072-674f9b4fb673 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:37.561273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a1666-e313-4078-ae00-c5d001371988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:37.561954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a1666-e313-4078-ae00-c5d001371988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:37.562237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a1666-e313-4078-ae00-c5d001371988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:37.562538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a1666-e313-4078-ae00-c5d001371988 admin admin] PATH_INFO: `/v3/users/53a8a3bfdaa644218ed405b066c7372a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:37.563762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a1666-e313-4078-ae00-c5d001371988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53a8a3bfdaa644218ed405b066c7372a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:37.565847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a1666-e313-4078-ae00-c5d001371988 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:37.610480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dc1a1666-e313-4078-ae00-c5d001371988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53a8a3bfdaa644218ed405b066c7372a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:37.617386 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dc1a1666-e313-4078-ae00-c5d001371988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53a8a3bfdaa644218ed405b066c7372a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:37.629698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dc1a1666-e313-4078-ae00-c5d001371988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53a8a3bfdaa644218ed405b066c7372a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:37.638798 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dc1a1666-e313-4078-ae00-c5d001371988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53a8a3bfdaa644218ed405b066c7372a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:37.639854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dc1a1666-e313-4078-ae00-c5d001371988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53a8a3bfdaa644218ed405b066c7372a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:37.646592 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1049/2084] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:37 2022] DELETE /identity/v3/users/53a8a3bfdaa644218ed405b066c7372a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:37.956828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:37.959699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:37.977815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f78d4a-83e1-4e37-8687-7dda852e6ace admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:38.040776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aed882-9926-4f4d-a064-15b42cd3a7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:38.041438 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aed882-9926-4f4d-a064-15b42cd3a7e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:38.041721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aed882-9926-4f4d-a064-15b42cd3a7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:38.041995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aed882-9926-4f4d-a064-15b42cd3a7e8 admin admin] PATH_INFO: `/v3/projects/ed2ebd80d3104853afea45a16496df14` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:38.043184 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aed882-9926-4f4d-a064-15b42cd3a7e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed2ebd80d3104853afea45a16496df14)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:38.045823 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aed882-9926-4f4d-a064-15b42cd3a7e8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:38.100179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-62aed882-9926-4f4d-a064-15b42cd3a7e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed2ebd80d3104853afea45a16496df14', 'request_id': 'req-62aed882-9926-4f4d-a064-15b42cd3a7e8'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:38.106096 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-62aed882-9926-4f4d-a064-15b42cd3a7e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed2ebd80d3104853afea45a16496df14', 'request_id': 'req-62aed882-9926-4f4d-a064-15b42cd3a7e8'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:38.117157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-62aed882-9926-4f4d-a064-15b42cd3a7e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed2ebd80d3104853afea45a16496df14', 'request_id': 'req-62aed882-9926-4f4d-a064-15b42cd3a7e8'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:38.129612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-62aed882-9926-4f4d-a064-15b42cd3a7e8 admin admin] The token cache is being invalidate because project ed2ebd80d3104853afea45a16496df14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:38.129901 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-62aed882-9926-4f4d-a064-15b42cd3a7e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:38.130557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-62aed882-9926-4f4d-a064-15b42cd3a7e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:38.132288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1036/2085] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:37 2022] DELETE /identity/v3/projects/ed2ebd80d3104853afea45a16496df14 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:38.137063 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be59fe-5544-435c-8c7c-f2f159c08dff None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:38.137530 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be59fe-5544-435c-8c7c-f2f159c08dff None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:38.137789 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be59fe-5544-435c-8c7c-f2f159c08dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:38.162564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-85be59fe-5544-435c-8c7c-f2f159c08dff None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:38.164101 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-85be59fe-5544-435c-8c7c-f2f159c08dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:38.215552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:38.230693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1050/2086] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:38.240637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269ff38-fdb9-4076-ba04-e9565a2c8ae5 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:38.240929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269ff38-fdb9-4076-ba04-e9565a2c8ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:38.241210 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269ff38-fdb9-4076-ba04-e9565a2c8ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:38.261315 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-a269ff38-fdb9-4076-ba04-e9565a2c8ae5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:38.262524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a269ff38-fdb9-4076-ba04-e9565a2c8ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:38.271004 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1037/2087] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:38.276680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be59fe-5544-435c-8c7c-f2f159c08dff None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:38.287140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be59fe-5544-435c-8c7c-f2f159c08dff None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:38.291726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7333f-2f61-4dd6-9b9d-4a87ea46b19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:38.292316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7333f-2f61-4dd6-9b9d-4a87ea46b19d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:38.292651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7333f-2f61-4dd6-9b9d-4a87ea46b19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:38.292928 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7333f-2f61-4dd6-9b9d-4a87ea46b19d admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:38.293569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7333f-2f61-4dd6-9b9d-4a87ea46b19d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:29:38.293871 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7333f-2f61-4dd6-9b9d-4a87ea46b19d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:38.295448 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7333f-2f61-4dd6-9b9d-4a87ea46b19d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:38.306807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1051/2088] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:29:38.311149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269ff38-fdb9-4076-ba04-e9565a2c8ae5 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:38.320484 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269ff38-fdb9-4076-ba04-e9565a2c8ae5 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:38.325087 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422f2df-c14f-4c9c-be17-4360ead40b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:38.325696 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422f2df-c14f-4c9c-be17-4360ead40b05 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:38.326468 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422f2df-c14f-4c9c-be17-4360ead40b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:38.326737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422f2df-c14f-4c9c-be17-4360ead40b05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:38.327368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422f2df-c14f-4c9c-be17-4360ead40b05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:38.329345 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422f2df-c14f-4c9c-be17-4360ead40b05 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:38.349023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:38.360376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1038/2089] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:38 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:38.364093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7333f-2f61-4dd6-9b9d-4a87ea46b19d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:38.377562 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7333f-2f61-4dd6-9b9d-4a87ea46b19d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:38.441020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c2dca-a4de-4ed4-8fdc-721f39c6e801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:38.441621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c2dca-a4de-4ed4-8fdc-721f39c6e801 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:38.441898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c2dca-a4de-4ed4-8fdc-721f39c6e801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:38.442166 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c2dca-a4de-4ed4-8fdc-721f39c6e801 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:38.442823 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c2dca-a4de-4ed4-8fdc-721f39c6e801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:38.444769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c2dca-a4de-4ed4-8fdc-721f39c6e801 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:38.466293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1052/2090] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:38 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:38.471053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422f2df-c14f-4c9c-be17-4360ead40b05 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:38.481326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422f2df-c14f-4c9c-be17-4360ead40b05 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:38.485963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4b800-a50d-4e6c-b427-294d2ff91d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:38.486551 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4b800-a50d-4e6c-b427-294d2ff91d25 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:38.486828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4b800-a50d-4e6c-b427-294d2ff91d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:38.487096 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4b800-a50d-4e6c-b427-294d2ff91d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:38.487782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4b800-a50d-4e6c-b427-294d2ff91d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:38.489374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4b800-a50d-4e6c-b427-294d2ff91d25 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:38.499695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1039/2091] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:38.503696 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c2dca-a4de-4ed4-8fdc-721f39c6e801 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:38.514024 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c2dca-a4de-4ed4-8fdc-721f39c6e801 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:38.518595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c1ae2-aec6-4e72-8de5-5c577d32c335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:38.519184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c1ae2-aec6-4e72-8de5-5c577d32c335 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:38.519465 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c1ae2-aec6-4e72-8de5-5c577d32c335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:38.519767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c1ae2-aec6-4e72-8de5-5c577d32c335 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:38.520417 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c1ae2-aec6-4e72-8de5-5c577d32c335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:38.521991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c1ae2-aec6-4e72-8de5-5c577d32c335 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:38.533285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1053/2092] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:38.537489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4b800-a50d-4e6c-b427-294d2ff91d25 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:38.547329 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4b800-a50d-4e6c-b427-294d2ff91d25 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:38.551913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd128f8-12ec-4dba-b331-1171269859ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:38.552553 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd128f8-12ec-4dba-b331-1171269859ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:38.552884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd128f8-12ec-4dba-b331-1171269859ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:38.553158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd128f8-12ec-4dba-b331-1171269859ac admin admin] PATH_INFO: `/v3/projects/b5841ef8313946afa4f0558df3ec1ada/users/8ec0813c2a29457db9517f9ebfa4b036/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:38.569147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd128f8-12ec-4dba-b331-1171269859ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5841ef8313946afa4f0558df3ec1ada, user_id=8ec0813c2a29457db9517f9ebfa4b036, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:38.571880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd128f8-12ec-4dba-b331-1171269859ac admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:38.585995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1040/2093] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:38 2022] PUT /identity/v3/projects/b5841ef8313946afa4f0558df3ec1ada/users/8ec0813c2a29457db9517f9ebfa4b036/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:38.597506 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffd818-c8c9-4b80-802b-be1015ec74d1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:38.597873 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffd818-c8c9-4b80-802b-be1015ec74d1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:38.598216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffd818-c8c9-4b80-802b-be1015ec74d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:38.627132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a8ffd818-c8c9-4b80-802b-be1015ec74d1 None None] MFA Rules not processed for user `8ec0813c2a29457db9517f9ebfa4b036`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:38.628415 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a8ffd818-c8c9-4b80-802b-be1015ec74d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:38.701794 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1054/2094] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:38.706472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd128f8-12ec-4dba-b331-1171269859ac admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:38.770094 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd128f8-12ec-4dba-b331-1171269859ac admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:38.774674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51288c52-6b92-4ecf-82e5-4b127a28427a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:38.775279 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51288c52-6b92-4ecf-82e5-4b127a28427a admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:38.775557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51288c52-6b92-4ecf-82e5-4b127a28427a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:38.775862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51288c52-6b92-4ecf-82e5-4b127a28427a admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:38.776489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51288c52-6b92-4ecf-82e5-4b127a28427a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:38.778393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51288c52-6b92-4ecf-82e5-4b127a28427a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:38.807311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1041/2095] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:38 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:38.811060 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffd818-c8c9-4b80-802b-be1015ec74d1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:38.821845 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffd818-c8c9-4b80-802b-be1015ec74d1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:38.882847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:38.898810 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e503b-1490-4a9e-9f5d-04dfa9c8c8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:38.899426 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e503b-1490-4a9e-9f5d-04dfa9c8c8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:38.899743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e503b-1490-4a9e-9f5d-04dfa9c8c8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:38.900018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e503b-1490-4a9e-9f5d-04dfa9c8c8c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:38.900676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e503b-1490-4a9e-9f5d-04dfa9c8c8c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:38.902600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e503b-1490-4a9e-9f5d-04dfa9c8c8c0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:38.931202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1055/2096] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:38 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:38.934542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51288c52-6b92-4ecf-82e5-4b127a28427a admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:38.948071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51288c52-6b92-4ecf-82e5-4b127a28427a admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:38.962128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45d657-8c26-420f-925d-41490bcb29f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:38.963273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45d657-8c26-420f-925d-41490bcb29f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:38.963644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45d657-8c26-420f-925d-41490bcb29f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:38.964199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45d657-8c26-420f-925d-41490bcb29f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:38.966856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45d657-8c26-420f-925d-41490bcb29f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:38.970870 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45d657-8c26-420f-925d-41490bcb29f1 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:38.994284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1042/2097] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:39.000034 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e503b-1490-4a9e-9f5d-04dfa9c8c8c0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:39.017251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e503b-1490-4a9e-9f5d-04dfa9c8c8c0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:39.022841 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157abe3-a491-41d6-b374-b3545d206dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:39.023438 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157abe3-a491-41d6-b374-b3545d206dff admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:39.023917 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157abe3-a491-41d6-b374-b3545d206dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:39.024207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157abe3-a491-41d6-b374-b3545d206dff admin admin] PATH_INFO: `/v3/projects/9129474538e144669e6468c8e5ffb686/users/cda22951dde34278b1532e61b92ea60c/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:39.042018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157abe3-a491-41d6-b374-b3545d206dff admin admin] RBAC: Authorizing `identity:create_grant(project_id=9129474538e144669e6468c8e5ffb686, user_id=cda22951dde34278b1532e61b92ea60c, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:39.045436 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157abe3-a491-41d6-b374-b3545d206dff admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:39.059503 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1056/2098] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:38 2022] PUT /identity/v3/projects/9129474538e144669e6468c8e5ffb686/users/cda22951dde34278b1532e61b92ea60c/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:39.064391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45d657-8c26-420f-925d-41490bcb29f1 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:39.075051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45d657-8c26-420f-925d-41490bcb29f1 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:39.140780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ff557-fd61-4f5b-9628-36694d73c6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:39.141549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ff557-fd61-4f5b-9628-36694d73c6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:39.141844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ff557-fd61-4f5b-9628-36694d73c6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:39.142115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ff557-fd61-4f5b-9628-36694d73c6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:39.142758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ff557-fd61-4f5b-9628-36694d73c6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:39.144496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ff557-fd61-4f5b-9628-36694d73c6a8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:39.156191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1043/2099] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:29:39.160467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157abe3-a491-41d6-b374-b3545d206dff admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:39.171561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157abe3-a491-41d6-b374-b3545d206dff admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:39.177003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86cb8d-6892-4afe-8d07-eef5c1ced019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:39.177711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86cb8d-6892-4afe-8d07-eef5c1ced019 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:39.177993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86cb8d-6892-4afe-8d07-eef5c1ced019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:39.178278 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86cb8d-6892-4afe-8d07-eef5c1ced019 admin admin] PATH_INFO: `/v3/projects/9129474538e144669e6468c8e5ffb686/users/cda22951dde34278b1532e61b92ea60c/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:39.180390 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86cb8d-6892-4afe-8d07-eef5c1ced019 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9129474538e144669e6468c8e5ffb686, user_id=cda22951dde34278b1532e61b92ea60c, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:39.183163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86cb8d-6892-4afe-8d07-eef5c1ced019 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:39.196952 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1057/2100] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:39 2022] PUT /identity/v3/projects/9129474538e144669e6468c8e5ffb686/users/cda22951dde34278b1532e61b92ea60c/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:39.211595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f7084-a810-4964-9e71-3cbd22798d22 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:39.212647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f7084-a810-4964-9e71-3cbd22798d22 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:39.212923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f7084-a810-4964-9e71-3cbd22798d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:39.280703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-896f7084-a810-4964-9e71-3cbd22798d22 None None] MFA Rules not processed for user `cda22951dde34278b1532e61b92ea60c`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:39.284873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-896f7084-a810-4964-9e71-3cbd22798d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:39.337074 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:39.472686 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1044/2101] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:39.485047 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86cb8d-6892-4afe-8d07-eef5c1ced019 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:39.486763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fc86cb8d-6892-4afe-8d07-eef5c1ced019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:39.567553 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86cb8d-6892-4afe-8d07-eef5c1ced019 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:39.571829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49ebcf-36af-4b3b-b959-5244c2ce257c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:39.572524 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49ebcf-36af-4b3b-b959-5244c2ce257c service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:39.572868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49ebcf-36af-4b3b-b959-5244c2ce257c service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:39.573814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49ebcf-36af-4b3b-b959-5244c2ce257c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:39.584925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49ebcf-36af-4b3b-b959-5244c2ce257c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:39.586367 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49ebcf-36af-4b3b-b959-5244c2ce257c service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:39.597445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1058/2102] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:39.691021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f7084-a810-4964-9e71-3cbd22798d22 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:39.700302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f7084-a810-4964-9e71-3cbd22798d22 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:39.704501 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab117125-facb-4329-affa-d7363e407cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:39.705090 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab117125-facb-4329-affa-d7363e407cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:39.705365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab117125-facb-4329-affa-d7363e407cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:39.705631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab117125-facb-4329-affa-d7363e407cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:39.764082 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab117125-facb-4329-affa-d7363e407cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:39.765528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab117125-facb-4329-affa-d7363e407cc2 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:39.774649 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1045/2103] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:40.041770 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49ebcf-36af-4b3b-b959-5244c2ce257c service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:40.105279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49ebcf-36af-4b3b-b959-5244c2ce257c service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:40.111125 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed19909-4309-4b29-a2e1-6766e7f08dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:40.112101 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed19909-4309-4b29-a2e1-6766e7f08dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:40.112599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed19909-4309-4b29-a2e1-6766e7f08dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:40.113102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed19909-4309-4b29-a2e1-6766e7f08dcb admin admin] PATH_INFO: `/v3/users/8ec0813c2a29457db9517f9ebfa4b036` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:40.114670 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed19909-4309-4b29-a2e1-6766e7f08dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ec0813c2a29457db9517f9ebfa4b036)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:40.117331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed19909-4309-4b29-a2e1-6766e7f08dcb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:40.161347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:40.162916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2ed19909-4309-4b29-a2e1-6766e7f08dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ec0813c2a29457db9517f9ebfa4b036'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.171359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2ed19909-4309-4b29-a2e1-6766e7f08dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ec0813c2a29457db9517f9ebfa4b036'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.182868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2ed19909-4309-4b29-a2e1-6766e7f08dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ec0813c2a29457db9517f9ebfa4b036'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.191370 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2ed19909-4309-4b29-a2e1-6766e7f08dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ec0813c2a29457db9517f9ebfa4b036'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.192322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2ed19909-4309-4b29-a2e1-6766e7f08dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ec0813c2a29457db9517f9ebfa4b036'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.200580 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1059/2104] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:40 2022] DELETE /identity/v3/users/8ec0813c2a29457db9517f9ebfa4b036 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:40.205549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d297f-433f-495b-8554-2c155e7faeb5 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:40.206017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d297f-433f-495b-8554-2c155e7faeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:40.206109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d297f-433f-495b-8554-2c155e7faeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:40.224988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-938d297f-433f-495b-8554-2c155e7faeb5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:40.226339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-938d297f-433f-495b-8554-2c155e7faeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:40.283712 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1046/2105] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:40.293149 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed19909-4309-4b29-a2e1-6766e7f08dcb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:40.294551 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2ed19909-4309-4b29-a2e1-6766e7f08dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:40.350648 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed19909-4309-4b29-a2e1-6766e7f08dcb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:40.354248 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d6346-4c5e-4c2b-a525-f2336cc1a27c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:40.354845 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d6346-4c5e-4c2b-a525-f2336cc1a27c service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:40.355132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d6346-4c5e-4c2b-a525-f2336cc1a27c service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:40.355401 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d6346-4c5e-4c2b-a525-f2336cc1a27c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:40.364436 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d6346-4c5e-4c2b-a525-f2336cc1a27c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:40.365852 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d6346-4c5e-4c2b-a525-f2336cc1a27c service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:40.375663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1060/2106] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:40.658425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d297f-433f-495b-8554-2c155e7faeb5 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:40.660025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-938d297f-433f-495b-8554-2c155e7faeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:40.681495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d297f-433f-495b-8554-2c155e7faeb5 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:40.686742 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:40.687392 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:40.687747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:40.687913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] PATH_INFO: `/v3/projects/b5841ef8313946afa4f0558df3ec1ada` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:40.689065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5841ef8313946afa4f0558df3ec1ada)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:40.691029 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:40.747173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5841ef8313946afa4f0558df3ec1ada', 'request_id': 'req-1aecf678-708a-48d2-b33c-477f770877dc'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.751433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:40.754309 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5841ef8313946afa4f0558df3ec1ada', 'request_id': 'req-1aecf678-708a-48d2-b33c-477f770877dc'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.765071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5841ef8313946afa4f0558df3ec1ada', 'request_id': 'req-1aecf678-708a-48d2-b33c-477f770877dc'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.780582 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] The token cache is being invalidate because project b5841ef8313946afa4f0558df3ec1ada was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:40.780880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.781562 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.786225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1047/2107] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:40 2022] DELETE /identity/v3/projects/b5841ef8313946afa4f0558df3ec1ada => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:40.787187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d6346-4c5e-4c2b-a525-f2336cc1a27c service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:40.797638 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d6346-4c5e-4c2b-a525-f2336cc1a27c service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:40.868550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e3f80-5cd5-4711-9d09-cfe90d56c072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:40.869196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e3f80-5cd5-4711-9d09-cfe90d56c072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:40.869469 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e3f80-5cd5-4711-9d09-cfe90d56c072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:40.869751 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e3f80-5cd5-4711-9d09-cfe90d56c072 admin admin] PATH_INFO: `/v3/users/cda22951dde34278b1532e61b92ea60c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:40.873599 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e3f80-5cd5-4711-9d09-cfe90d56c072 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cda22951dde34278b1532e61b92ea60c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:40.875701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e3f80-5cd5-4711-9d09-cfe90d56c072 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:40.945392 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad1e3f80-5cd5-4711-9d09-cfe90d56c072 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cda22951dde34278b1532e61b92ea60c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.953521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad1e3f80-5cd5-4711-9d09-cfe90d56c072 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cda22951dde34278b1532e61b92ea60c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.962722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:40.974472 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad1e3f80-5cd5-4711-9d09-cfe90d56c072 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cda22951dde34278b1532e61b92ea60c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.984662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad1e3f80-5cd5-4711-9d09-cfe90d56c072 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cda22951dde34278b1532e61b92ea60c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.985360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad1e3f80-5cd5-4711-9d09-cfe90d56c072 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cda22951dde34278b1532e61b92ea60c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:40.995352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1061/2108] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:29:40 2022] DELETE /identity/v3/users/cda22951dde34278b1532e61b92ea60c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:41.334741 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:41.336613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:41.349956 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecf678-708a-48d2-b33c-477f770877dc admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:41.406535 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99ec49-57b2-4195-84d3-239ec865abd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:41.407158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99ec49-57b2-4195-84d3-239ec865abd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:41.407437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99ec49-57b2-4195-84d3-239ec865abd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:41.407745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99ec49-57b2-4195-84d3-239ec865abd3 admin admin] PATH_INFO: `/v3/projects/9129474538e144669e6468c8e5ffb686` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:41.408865 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99ec49-57b2-4195-84d3-239ec865abd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9129474538e144669e6468c8e5ffb686)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:41.410754 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99ec49-57b2-4195-84d3-239ec865abd3 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:41.457204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-fd99ec49-57b2-4195-84d3-239ec865abd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9129474538e144669e6468c8e5ffb686', 'request_id': 'req-fd99ec49-57b2-4195-84d3-239ec865abd3'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:41.464120 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-fd99ec49-57b2-4195-84d3-239ec865abd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9129474538e144669e6468c8e5ffb686', 'request_id': 'req-fd99ec49-57b2-4195-84d3-239ec865abd3'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:41.472616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-fd99ec49-57b2-4195-84d3-239ec865abd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9129474538e144669e6468c8e5ffb686', 'request_id': 'req-fd99ec49-57b2-4195-84d3-239ec865abd3'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:41.485054 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-fd99ec49-57b2-4195-84d3-239ec865abd3 admin admin] The token cache is being invalidate because project 9129474538e144669e6468c8e5ffb686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:29:41.485343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-fd99ec49-57b2-4195-84d3-239ec865abd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:41.485995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-fd99ec49-57b2-4195-84d3-239ec865abd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:29:41.487527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1048/2109] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:29:41 2022] DELETE /identity/v3/projects/9129474538e144669e6468c8e5ffb686 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:29:41.493336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712488a2-23fb-47d8-bbbe-6e6a08aae7e3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:41.493863 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712488a2-23fb-47d8-bbbe-6e6a08aae7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:41.494092 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712488a2-23fb-47d8-bbbe-6e6a08aae7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:41.515688 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-712488a2-23fb-47d8-bbbe-6e6a08aae7e3 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:41.516919 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-712488a2-23fb-47d8-bbbe-6e6a08aae7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:41.582702 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1062/2110] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:41.592480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacafc3e-9e20-42f2-aa27-118d64f8bddc None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:41.592957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacafc3e-9e20-42f2-aa27-118d64f8bddc None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:41.593156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacafc3e-9e20-42f2-aa27-118d64f8bddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:41.614538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-bacafc3e-9e20-42f2-aa27-118d64f8bddc None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:41.615692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bacafc3e-9e20-42f2-aa27-118d64f8bddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:41.624868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1049/2111] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:29:41.629303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712488a2-23fb-47d8-bbbe-6e6a08aae7e3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:41.639673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712488a2-23fb-47d8-bbbe-6e6a08aae7e3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:41.644323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a927df-54d1-4a08-9fcc-ceb3692ef41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:41.644919 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a927df-54d1-4a08-9fcc-ceb3692ef41d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:41.645198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a927df-54d1-4a08-9fcc-ceb3692ef41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:41.645546 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a927df-54d1-4a08-9fcc-ceb3692ef41d admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:41.646160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a927df-54d1-4a08-9fcc-ceb3692ef41d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:29:41.646559 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a927df-54d1-4a08-9fcc-ceb3692ef41d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:41.648493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a927df-54d1-4a08-9fcc-ceb3692ef41d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:41.662348 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1063/2112] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:29:41.666601 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacafc3e-9e20-42f2-aa27-118d64f8bddc None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:41.677593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacafc3e-9e20-42f2-aa27-118d64f8bddc None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:41.684577 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72a09e-61ab-4747-8171-72b40acdcda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:41.685301 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72a09e-61ab-4747-8171-72b40acdcda8 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:41.685706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72a09e-61ab-4747-8171-72b40acdcda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:41.686153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72a09e-61ab-4747-8171-72b40acdcda8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:41.686916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72a09e-61ab-4747-8171-72b40acdcda8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:41.689084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72a09e-61ab-4747-8171-72b40acdcda8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:41.718246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1050/2113] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:29:41 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:41.722435 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a927df-54d1-4a08-9fcc-ceb3692ef41d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:41.733716 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a927df-54d1-4a08-9fcc-ceb3692ef41d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:41.798585 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846aad26-9e25-4af1-8ce4-a553921ea977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:41.799197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846aad26-9e25-4af1-8ce4-a553921ea977 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:41.799474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846aad26-9e25-4af1-8ce4-a553921ea977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:41.799774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846aad26-9e25-4af1-8ce4-a553921ea977 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:41.800403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846aad26-9e25-4af1-8ce4-a553921ea977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:41.802348 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846aad26-9e25-4af1-8ce4-a553921ea977 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:41.826344 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1064/2114] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:29:41 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:29:41.830154 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72a09e-61ab-4747-8171-72b40acdcda8 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:41.840893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72a09e-61ab-4747-8171-72b40acdcda8 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:41.845952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf940bb-6855-4a38-bdb3-277c6da25bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:41.846714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf940bb-6855-4a38-bdb3-277c6da25bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:41.846996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf940bb-6855-4a38-bdb3-277c6da25bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:41.847270 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf940bb-6855-4a38-bdb3-277c6da25bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:41.848031 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf940bb-6855-4a38-bdb3-277c6da25bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:41.849826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf940bb-6855-4a38-bdb3-277c6da25bf7 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:41.861047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1051/2115] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:41.865169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846aad26-9e25-4af1-8ce4-a553921ea977 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:41.875782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846aad26-9e25-4af1-8ce4-a553921ea977 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:41.882287 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278bae9-ad04-4674-8c18-26e22a8b6d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:41.882837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278bae9-ad04-4674-8c18-26e22a8b6d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:41.883113 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278bae9-ad04-4674-8c18-26e22a8b6d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:41.883506 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278bae9-ad04-4674-8c18-26e22a8b6d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:41.884194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278bae9-ad04-4674-8c18-26e22a8b6d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:41.885891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278bae9-ad04-4674-8c18-26e22a8b6d19 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:41.897923 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1065/2116] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:29:41.902073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf940bb-6855-4a38-bdb3-277c6da25bf7 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:41.929805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf940bb-6855-4a38-bdb3-277c6da25bf7 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:41.938011 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca829e-1944-4539-a181-2412db345839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:41.938310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca829e-1944-4539-a181-2412db345839 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:41.938607 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca829e-1944-4539-a181-2412db345839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:41.938890 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca829e-1944-4539-a181-2412db345839 admin admin] PATH_INFO: `/v3/projects/882b096f880e450d843e7c94dc63b26c/users/5a6718b4da5a4133b1d1a930c97ec560/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:41.956838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca829e-1944-4539-a181-2412db345839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=882b096f880e450d843e7c94dc63b26c, user_id=5a6718b4da5a4133b1d1a930c97ec560, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:41.959649 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca829e-1944-4539-a181-2412db345839 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:41.979613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1052/2117] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:29:41 2022] PUT /identity/v3/projects/882b096f880e450d843e7c94dc63b26c/users/5a6718b4da5a4133b1d1a930c97ec560/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:29:41.987268 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6b5fa-8525-4cc7-a288-0c1f16d87243 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:41.987552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6b5fa-8525-4cc7-a288-0c1f16d87243 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:41.987860 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6b5fa-8525-4cc7-a288-0c1f16d87243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:42.019769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8ac6b5fa-8525-4cc7-a288-0c1f16d87243 None None] MFA Rules not processed for user `5a6718b4da5a4133b1d1a930c97ec560`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:29:42.021061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8ac6b5fa-8525-4cc7-a288-0c1f16d87243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:42.071299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:42.105698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1066/2118] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:29:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:29:42.117776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca829e-1944-4539-a181-2412db345839 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:29:42.119516 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-39ca829e-1944-4539-a181-2412db345839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:29:42.130481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:42.187158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca829e-1944-4539-a181-2412db345839 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:29:42.190913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc358a8-4ab2-40bc-9b1a-963a499c53cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:29:42.191544 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc358a8-4ab2-40bc-9b1a-963a499c53cd service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:29:42.191847 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc358a8-4ab2-40bc-9b1a-963a499c53cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:29:42.192110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc358a8-4ab2-40bc-9b1a-963a499c53cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:29:42.200943 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc358a8-4ab2-40bc-9b1a-963a499c53cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:29:42.202381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc358a8-4ab2-40bc-9b1a-963a499c53cd service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:29:42.219045 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1053/2119] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:29:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:29:52.102651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=9 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:29:52.213153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=24 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:03.620965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06ce70-e950-495c-9dde-4da6c58ce9bb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:03.621404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06ce70-e950-495c-9dde-4da6c58ce9bb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:03.621537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06ce70-e950-495c-9dde-4da6c58ce9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:03.642475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9f06ce70-e950-495c-9dde-4da6c58ce9bb None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:03.644227 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9f06ce70-e950-495c-9dde-4da6c58ce9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:03.715860 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1067/2120] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:30:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:03.767414 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc358a8-4ab2-40bc-9b1a-963a499c53cd service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:03.768948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-fbc358a8-4ab2-40bc-9b1a-963a499c53cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:03.844953 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc358a8-4ab2-40bc-9b1a-963a499c53cd service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:03.851166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f92b1-d822-440b-a674-0fa6337c23f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:03.852679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f92b1-d822-440b-a674-0fa6337c23f5 service swift] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:03.852679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f92b1-d822-440b-a674-0fa6337c23f5 service swift] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:03.852679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f92b1-d822-440b-a674-0fa6337c23f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:03.862200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f92b1-d822-440b-a674-0fa6337c23f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:03.863973 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f92b1-d822-440b-a674-0fa6337c23f5 service swift] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:03.887908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1054/2121] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:30:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:30:04.569528 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d01f6b-7845-4f7d-8cd3-9f7de56877dc None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:04.569911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d01f6b-7845-4f7d-8cd3-9f7de56877dc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:04.570186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d01f6b-7845-4f7d-8cd3-9f7de56877dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:04.591313 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-17d01f6b-7845-4f7d-8cd3-9f7de56877dc None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:04.593185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-17d01f6b-7845-4f7d-8cd3-9f7de56877dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:04.602624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1068/2122] 10.222.0.27 () {58 vars in 967 bytes} [Mon Sep 12 23:30:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:04.616858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f92b1-d822-440b-a674-0fa6337c23f5 service swift] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:04.631153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f92b1-d822-440b-a674-0fa6337c23f5 service swift] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:04.637674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b314ba1-a5ea-4613-be23-526575220695 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:04.639242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b314ba1-a5ea-4613-be23-526575220695 service swift] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:04.639631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b314ba1-a5ea-4613-be23-526575220695 service swift] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:04.640249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b314ba1-a5ea-4613-be23-526575220695 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:04.651413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b314ba1-a5ea-4613-be23-526575220695 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:04.653200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b314ba1-a5ea-4613-be23-526575220695 service swift] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:04.675268 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1055/2123] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:30:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:30:05.041885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcc521-9bd3-486e-b807-5f5985fb3aea None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:05.043387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcc521-9bd3-486e-b807-5f5985fb3aea None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:05.043963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcc521-9bd3-486e-b807-5f5985fb3aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:05.110499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d1dcc521-9bd3-486e-b807-5f5985fb3aea None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:05.116069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d1dcc521-9bd3-486e-b807-5f5985fb3aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:05.126755 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1069/2124] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:30:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:05.168114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b314ba1-a5ea-4613-be23-526575220695 service swift] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:05.185077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b314ba1-a5ea-4613-be23-526575220695 service swift] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:05.190933 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47191a-350c-4bee-b4b0-605b4001b769 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:05.191685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47191a-350c-4bee-b4b0-605b4001b769 service swift] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:05.192054 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47191a-350c-4bee-b4b0-605b4001b769 service swift] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:05.192396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47191a-350c-4bee-b4b0-605b4001b769 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:05.201947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47191a-350c-4bee-b4b0-605b4001b769 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:05.203531 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47191a-350c-4bee-b4b0-605b4001b769 service swift] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:05.213050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1056/2125] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:30:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:30:05.586190 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabf7fd-96b8-4a72-8570-2c9787943198 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:05.586480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabf7fd-96b8-4a72-8570-2c9787943198 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:05.586752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabf7fd-96b8-4a72-8570-2c9787943198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:05.611489 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9dabf7fd-96b8-4a72-8570-2c9787943198 None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:05.613480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9dabf7fd-96b8-4a72-8570-2c9787943198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:05.629090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1070/2126] 10.222.0.27 () {58 vars in 967 bytes} [Mon Sep 12 23:30:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:30:05.644018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47191a-350c-4bee-b4b0-605b4001b769 service swift] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:05.664431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47191a-350c-4bee-b4b0-605b4001b769 service swift] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:05.670410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c123ff-669f-43ca-a6e3-9031a5a15149 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:05.671214 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c123ff-669f-43ca-a6e3-9031a5a15149 service swift] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:05.671499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c123ff-669f-43ca-a6e3-9031a5a15149 service swift] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:05.671808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c123ff-669f-43ca-a6e3-9031a5a15149 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:05.680483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c123ff-669f-43ca-a6e3-9031a5a15149 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:05.682093 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c123ff-669f-43ca-a6e3-9031a5a15149 service swift] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:05.690501 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1057/2127] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:30:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 23:30:05.740106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabf7fd-96b8-4a72-8570-2c9787943198 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:05.741896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9dabf7fd-96b8-4a72-8570-2c9787943198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:05.810194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabf7fd-96b8-4a72-8570-2c9787943198 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:05.814729 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44480fff-3b50-4088-a530-04507a0c6511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:05.815359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44480fff-3b50-4088-a530-04507a0c6511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:05.815684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44480fff-3b50-4088-a530-04507a0c6511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:05.815959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44480fff-3b50-4088-a530-04507a0c6511 admin admin] PATH_INFO: `/v3/users/638ad311373f44edbccb5b6c3fa971c0` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:05.817112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44480fff-3b50-4088-a530-04507a0c6511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=638ad311373f44edbccb5b6c3fa971c0)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:05.818995 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44480fff-3b50-4088-a530-04507a0c6511 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:05.871098 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-44480fff-3b50-4088-a530-04507a0c6511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '638ad311373f44edbccb5b6c3fa971c0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:05.879049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-44480fff-3b50-4088-a530-04507a0c6511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '638ad311373f44edbccb5b6c3fa971c0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:05.894113 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-44480fff-3b50-4088-a530-04507a0c6511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '638ad311373f44edbccb5b6c3fa971c0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:05.902208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-44480fff-3b50-4088-a530-04507a0c6511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '638ad311373f44edbccb5b6c3fa971c0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:05.902927 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-44480fff-3b50-4088-a530-04507a0c6511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '638ad311373f44edbccb5b6c3fa971c0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:05.916206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1071/2128] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:05 2022] DELETE /identity/v3/users/638ad311373f44edbccb5b6c3fa971c0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:05.922570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e18f05-3e22-4838-9638-911e3a18a06b None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:05.922979 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e18f05-3e22-4838-9638-911e3a18a06b None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:05.923446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e18f05-3e22-4838-9638-911e3a18a06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:05.945069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-f5e18f05-3e22-4838-9638-911e3a18a06b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:05.946441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-f5e18f05-3e22-4838-9638-911e3a18a06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:06.014243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1058/2129] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:06.027921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44480fff-3b50-4088-a530-04507a0c6511 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:06.029697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-44480fff-3b50-4088-a530-04507a0c6511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:06.037202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:06.115411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44480fff-3b50-4088-a530-04507a0c6511 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:06.133855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1aa7f-3fa7-49fc-97ad-e998d3294462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:06.134587 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1aa7f-3fa7-49fc-97ad-e998d3294462 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:06.134960 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1aa7f-3fa7-49fc-97ad-e998d3294462 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:06.135296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1aa7f-3fa7-49fc-97ad-e998d3294462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:06.148518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1aa7f-3fa7-49fc-97ad-e998d3294462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:06.153091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1aa7f-3fa7-49fc-97ad-e998d3294462 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:06.164115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1072/2130] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:06.479960 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e18f05-3e22-4838-9638-911e3a18a06b None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:06.481490 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-f5e18f05-3e22-4838-9638-911e3a18a06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:06.505719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e18f05-3e22-4838-9638-911e3a18a06b None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:06.512879 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:06.514202 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:06.514708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:06.514987 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] PATH_INFO: `/v3/projects/a54f57b551e8417490cd01e11967db9d` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:06.516199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a54f57b551e8417490cd01e11967db9d)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:06.518213 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:06.568438 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:06.571842 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a54f57b551e8417490cd01e11967db9d', 'request_id': 'req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:06.579763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a54f57b551e8417490cd01e11967db9d', 'request_id': 'req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:06.586579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a54f57b551e8417490cd01e11967db9d', 'request_id': 'req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:06.598389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] The token cache is being invalidate because project a54f57b551e8417490cd01e11967db9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:06.598678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:06.599312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:06.600826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1059/2131] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:06 2022] DELETE /identity/v3/projects/a54f57b551e8417490cd01e11967db9d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:06.604883 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1aa7f-3fa7-49fc-97ad-e998d3294462 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:06.615725 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1aa7f-3fa7-49fc-97ad-e998d3294462 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:06.700112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4a17e-aec9-4778-a5b5-b22a2d6b0f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:06.700875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4a17e-aec9-4778-a5b5-b22a2d6b0f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:06.701776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4a17e-aec9-4778-a5b5-b22a2d6b0f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:06.702153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4a17e-aec9-4778-a5b5-b22a2d6b0f36 admin admin] PATH_INFO: `/v3/users/62a9ea84ff20455996860080dd31d3d0` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:06.703340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4a17e-aec9-4778-a5b5-b22a2d6b0f36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62a9ea84ff20455996860080dd31d3d0)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:06.708626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4a17e-aec9-4778-a5b5-b22a2d6b0f36 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:06.771298 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-36d4a17e-aec9-4778-a5b5-b22a2d6b0f36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62a9ea84ff20455996860080dd31d3d0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:06.779560 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-36d4a17e-aec9-4778-a5b5-b22a2d6b0f36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62a9ea84ff20455996860080dd31d3d0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:06.792881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-36d4a17e-aec9-4778-a5b5-b22a2d6b0f36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62a9ea84ff20455996860080dd31d3d0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:06.802615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-36d4a17e-aec9-4778-a5b5-b22a2d6b0f36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62a9ea84ff20455996860080dd31d3d0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:06.803687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-36d4a17e-aec9-4778-a5b5-b22a2d6b0f36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62a9ea84ff20455996860080dd31d3d0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:06.810570 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1073/2132] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:06 2022] DELETE /identity/v3/users/62a9ea84ff20455996860080dd31d3d0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:07.077130 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:07.078641 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:07.091915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65f9fe-d515-48fe-b62a-2a2f4eafc946 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:07.164167 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf090623-560e-46d9-91fd-7576060a1d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:07.164808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf090623-560e-46d9-91fd-7576060a1d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:07.165083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf090623-560e-46d9-91fd-7576060a1d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:07.165355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf090623-560e-46d9-91fd-7576060a1d3f admin admin] PATH_INFO: `/v3/projects/8dd90954a4984344b7b11250d749b357` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:07.166469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf090623-560e-46d9-91fd-7576060a1d3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd90954a4984344b7b11250d749b357)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:07.169127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf090623-560e-46d9-91fd-7576060a1d3f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:07.226710 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-bf090623-560e-46d9-91fd-7576060a1d3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd90954a4984344b7b11250d749b357', 'request_id': 'req-bf090623-560e-46d9-91fd-7576060a1d3f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:07.232062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-bf090623-560e-46d9-91fd-7576060a1d3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd90954a4984344b7b11250d749b357', 'request_id': 'req-bf090623-560e-46d9-91fd-7576060a1d3f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:07.240225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-bf090623-560e-46d9-91fd-7576060a1d3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd90954a4984344b7b11250d749b357', 'request_id': 'req-bf090623-560e-46d9-91fd-7576060a1d3f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:07.251713 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-bf090623-560e-46d9-91fd-7576060a1d3f admin admin] The token cache is being invalidate because project 8dd90954a4984344b7b11250d749b357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:07.251999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-bf090623-560e-46d9-91fd-7576060a1d3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:07.252660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-bf090623-560e-46d9-91fd-7576060a1d3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:07.254207 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1060/2133] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:07 2022] DELETE /identity/v3/projects/8dd90954a4984344b7b11250d749b357 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:07.259070 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c7fdf-6426-432c-8214-52939383779d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:07.259519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c7fdf-6426-432c-8214-52939383779d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:07.259647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c7fdf-6426-432c-8214-52939383779d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:07.281120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-302c7fdf-6426-432c-8214-52939383779d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:07.282889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-302c7fdf-6426-432c-8214-52939383779d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:07.303165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:07.345183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1074/2134] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:07.352875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ba0c6-43d5-4448-ae11-dc5bed1bdb47 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:07.353164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ba0c6-43d5-4448-ae11-dc5bed1bdb47 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:07.353437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ba0c6-43d5-4448-ae11-dc5bed1bdb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:07.374694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-282ba0c6-43d5-4448-ae11-dc5bed1bdb47 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:07.376107 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-282ba0c6-43d5-4448-ae11-dc5bed1bdb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:07.384212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1061/2135] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:07.389088 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c7fdf-6426-432c-8214-52939383779d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:07.398917 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c7fdf-6426-432c-8214-52939383779d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:07.403631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0b94b-4b6b-423a-a3d1-506799b4e0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:07.404672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0b94b-4b6b-423a-a3d1-506799b4e0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:07.405154 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0b94b-4b6b-423a-a3d1-506799b4e0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:07.405446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0b94b-4b6b-423a-a3d1-506799b4e0d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:07.406093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0b94b-4b6b-423a-a3d1-506799b4e0d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:30:07.407986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0b94b-4b6b-423a-a3d1-506799b4e0d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:07.409627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0b94b-4b6b-423a-a3d1-506799b4e0d8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:07.423738 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1075/2136] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:30:07.426772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ba0c6-43d5-4448-ae11-dc5bed1bdb47 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:07.438181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ba0c6-43d5-4448-ae11-dc5bed1bdb47 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:07.443937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24807318-aff1-4c38-a83f-de00a4257b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:07.444603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24807318-aff1-4c38-a83f-de00a4257b85 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:07.444972 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24807318-aff1-4c38-a83f-de00a4257b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:07.445295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24807318-aff1-4c38-a83f-de00a4257b85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:07.445979 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24807318-aff1-4c38-a83f-de00a4257b85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:07.455756 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24807318-aff1-4c38-a83f-de00a4257b85 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:07.486926 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1062/2137] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:30:07 2022] POST /identity/v3/projects => generated 401 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:07.490061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0b94b-4b6b-423a-a3d1-506799b4e0d8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:07.500129 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0b94b-4b6b-423a-a3d1-506799b4e0d8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:07.555097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a96bcc-2be5-4117-84c9-1479d3a9e2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:07.555715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a96bcc-2be5-4117-84c9-1479d3a9e2cd admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:07.555995 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a96bcc-2be5-4117-84c9-1479d3a9e2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:07.556263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a96bcc-2be5-4117-84c9-1479d3a9e2cd admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:07.556898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a96bcc-2be5-4117-84c9-1479d3a9e2cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:07.558776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a96bcc-2be5-4117-84c9-1479d3a9e2cd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:07.579393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1076/2138] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:30:07 2022] POST /identity/v3/users => generated 301 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:07.583510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24807318-aff1-4c38-a83f-de00a4257b85 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:07.593514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24807318-aff1-4c38-a83f-de00a4257b85 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:07.598100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2f0fd-6f59-40bb-a879-9f956cefdc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:07.598634 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2f0fd-6f59-40bb-a879-9f956cefdc92 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:07.598910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2f0fd-6f59-40bb-a879-9f956cefdc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:07.599178 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2f0fd-6f59-40bb-a879-9f956cefdc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:07.599844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2f0fd-6f59-40bb-a879-9f956cefdc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:07.601431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2f0fd-6f59-40bb-a879-9f956cefdc92 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:07.611587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1063/2139] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:07.615307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a96bcc-2be5-4117-84c9-1479d3a9e2cd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:07.625160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a96bcc-2be5-4117-84c9-1479d3a9e2cd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:07.629735 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea89c64-a6c8-4bf9-afd2-092a04be6cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:07.630317 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea89c64-a6c8-4bf9-afd2-092a04be6cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:07.630598 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea89c64-a6c8-4bf9-afd2-092a04be6cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:07.630871 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea89c64-a6c8-4bf9-afd2-092a04be6cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:07.631506 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea89c64-a6c8-4bf9-afd2-092a04be6cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:07.633100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea89c64-a6c8-4bf9-afd2-092a04be6cb9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:07.643884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1077/2140] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:07.648001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2f0fd-6f59-40bb-a879-9f956cefdc92 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:07.658312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2f0fd-6f59-40bb-a879-9f956cefdc92 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:07.662873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839c6e-30c5-4603-a475-a6b2bc7dbc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:07.663586 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839c6e-30c5-4603-a475-a6b2bc7dbc77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:07.663899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839c6e-30c5-4603-a475-a6b2bc7dbc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:07.664175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839c6e-30c5-4603-a475-a6b2bc7dbc77 admin admin] PATH_INFO: `/v3/projects/d1d87447cbcc42779b39aaaa511c72e5/users/41d457c7846340c79e35b441a8ec4cbd/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:07.679266 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49839c6e-30c5-4603-a475-a6b2bc7dbc77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d87447cbcc42779b39aaaa511c72e5, user_id=41d457c7846340c79e35b441a8ec4cbd, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:07.681926 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49839c6e-30c5-4603-a475-a6b2bc7dbc77 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:07.690304 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:07.695420 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1064/2141] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:07 2022] PUT /identity/v3/projects/d1d87447cbcc42779b39aaaa511c72e5/users/41d457c7846340c79e35b441a8ec4cbd/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:07.706365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acae7af-e2e7-4013-9b32-92145f9aac89 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:07.706657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acae7af-e2e7-4013-9b32-92145f9aac89 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:07.706938 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acae7af-e2e7-4013-9b32-92145f9aac89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:07.734613 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4acae7af-e2e7-4013-9b32-92145f9aac89 None None] MFA Rules not processed for user `41d457c7846340c79e35b441a8ec4cbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:07.735856 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4acae7af-e2e7-4013-9b32-92145f9aac89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:07.797871 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1078/2142] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:07 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:07.802483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839c6e-30c5-4603-a475-a6b2bc7dbc77 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:07.864453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839c6e-30c5-4603-a475-a6b2bc7dbc77 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:07.869403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007a8ea-5be0-4e14-9aa8-3266f80850cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:07.870014 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007a8ea-5be0-4e14-9aa8-3266f80850cb admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:07.870297 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007a8ea-5be0-4e14-9aa8-3266f80850cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:07.870571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007a8ea-5be0-4e14-9aa8-3266f80850cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:07.871216 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007a8ea-5be0-4e14-9aa8-3266f80850cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:07.873665 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007a8ea-5be0-4e14-9aa8-3266f80850cb admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:07.902768 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1065/2143] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:30:07 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:07.907332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acae7af-e2e7-4013-9b32-92145f9aac89 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:07.926394 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acae7af-e2e7-4013-9b32-92145f9aac89 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:07.945731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:07.987389 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543a8b8-bcf5-4c7a-862d-8f23e8bc56be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:07.988059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543a8b8-bcf5-4c7a-862d-8f23e8bc56be admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:07.988343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543a8b8-bcf5-4c7a-862d-8f23e8bc56be admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:07.988617 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543a8b8-bcf5-4c7a-862d-8f23e8bc56be admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:07.989257 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543a8b8-bcf5-4c7a-862d-8f23e8bc56be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:07.991146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543a8b8-bcf5-4c7a-862d-8f23e8bc56be admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:08.017765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1079/2144] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:30:07 2022] POST /identity/v3/users => generated 307 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:30:08.021993 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007a8ea-5be0-4e14-9aa8-3266f80850cb admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:08.033458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007a8ea-5be0-4e14-9aa8-3266f80850cb admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:08.038309 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be3211-5227-460a-8d8e-84690d5e84dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:08.038915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be3211-5227-460a-8d8e-84690d5e84dc admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:08.039192 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be3211-5227-460a-8d8e-84690d5e84dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:08.039462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be3211-5227-460a-8d8e-84690d5e84dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:08.040232 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be3211-5227-460a-8d8e-84690d5e84dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:08.041815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be3211-5227-460a-8d8e-84690d5e84dc admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:08.056709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1066/2145] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:08.062039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543a8b8-bcf5-4c7a-862d-8f23e8bc56be admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:08.072541 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543a8b8-bcf5-4c7a-862d-8f23e8bc56be admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:08.078426 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6770bb-1bcd-41dd-aba2-b383c01440f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:08.079090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6770bb-1bcd-41dd-aba2-b383c01440f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:08.079811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6770bb-1bcd-41dd-aba2-b383c01440f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:08.079811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6770bb-1bcd-41dd-aba2-b383c01440f5 admin admin] PATH_INFO: `/v3/projects/8a37123d72eb402f96aabc0f1aaa7239/users/41e85af97a8249f1b81f263deff04f6e/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:08.096530 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6770bb-1bcd-41dd-aba2-b383c01440f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a37123d72eb402f96aabc0f1aaa7239, user_id=41e85af97a8249f1b81f263deff04f6e, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:08.099146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6770bb-1bcd-41dd-aba2-b383c01440f5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:08.112912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1080/2146] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:08 2022] PUT /identity/v3/projects/8a37123d72eb402f96aabc0f1aaa7239/users/41e85af97a8249f1b81f263deff04f6e/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:30:08.116611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be3211-5227-460a-8d8e-84690d5e84dc admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:08.126869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be3211-5227-460a-8d8e-84690d5e84dc admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:08.189533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccda02-98df-4440-8101-37de7b4bef01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:08.190133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccda02-98df-4440-8101-37de7b4bef01 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:08.190412 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccda02-98df-4440-8101-37de7b4bef01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:08.190686 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccda02-98df-4440-8101-37de7b4bef01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:08.191335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccda02-98df-4440-8101-37de7b4bef01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:08.192984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccda02-98df-4440-8101-37de7b4bef01 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:08.207117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1067/2147] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:08.208428 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6770bb-1bcd-41dd-aba2-b383c01440f5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:08.222248 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6770bb-1bcd-41dd-aba2-b383c01440f5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:08.227314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d48a3f-1619-4648-a893-86b201da05d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:08.228108 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d48a3f-1619-4648-a893-86b201da05d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:08.228384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d48a3f-1619-4648-a893-86b201da05d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:08.228665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d48a3f-1619-4648-a893-86b201da05d1 admin admin] PATH_INFO: `/v3/projects/8a37123d72eb402f96aabc0f1aaa7239/users/41e85af97a8249f1b81f263deff04f6e/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:08.230619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d48a3f-1619-4648-a893-86b201da05d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a37123d72eb402f96aabc0f1aaa7239, user_id=41e85af97a8249f1b81f263deff04f6e, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:08.234294 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d48a3f-1619-4648-a893-86b201da05d1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:08.245296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1081/2148] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:08 2022] PUT /identity/v3/projects/8a37123d72eb402f96aabc0f1aaa7239/users/41e85af97a8249f1b81f263deff04f6e/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:30:08.256280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7c3b2-1e55-45dc-9c35-20c00e0cd7e6 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:08.256589 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7c3b2-1e55-45dc-9c35-20c00e0cd7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:08.256899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7c3b2-1e55-45dc-9c35-20c00e0cd7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:08.287265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-7bc7c3b2-1e55-45dc-9c35-20c00e0cd7e6 None None] MFA Rules not processed for user `41e85af97a8249f1b81f263deff04f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:08.288761 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7bc7c3b2-1e55-45dc-9c35-20c00e0cd7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:08.349650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:08.364680 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1068/2149] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:08 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:08.372736 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d48a3f-1619-4648-a893-86b201da05d1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:08.375085 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e3d48a3f-1619-4648-a893-86b201da05d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:08.446349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d48a3f-1619-4648-a893-86b201da05d1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:08.450011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e96c5-da62-43af-90b4-fb3bacb14b43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:08.450612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e96c5-da62-43af-90b4-fb3bacb14b43 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:08.450933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e96c5-da62-43af-90b4-fb3bacb14b43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:08.451206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e96c5-da62-43af-90b4-fb3bacb14b43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:08.461915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e96c5-da62-43af-90b4-fb3bacb14b43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:08.463353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e96c5-da62-43af-90b4-fb3bacb14b43 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:08.473842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1082/2150] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:08 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:08.565402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7c3b2-1e55-45dc-9c35-20c00e0cd7e6 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:08.567164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7bc7c3b2-1e55-45dc-9c35-20c00e0cd7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:08.579088 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7c3b2-1e55-45dc-9c35-20c00e0cd7e6 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:08.583290 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f298f2-a048-4cb9-914f-b534a176c877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:08.583477 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f298f2-a048-4cb9-914f-b534a176c877 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:08.583786 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f298f2-a048-4cb9-914f-b534a176c877 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:08.584056 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f298f2-a048-4cb9-914f-b534a176c877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:08.660036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f298f2-a048-4cb9-914f-b534a176c877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:08.661585 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f298f2-a048-4cb9-914f-b534a176c877 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:08.672771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1069/2151] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:08 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:09.561046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97da38-64be-43e0-802e-ac9232e18bc3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:09.561494 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97da38-64be-43e0-802e-ac9232e18bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:09.561634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97da38-64be-43e0-802e-ac9232e18bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:09.583785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2e97da38-64be-43e0-802e-ac9232e18bc3 None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:09.584905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2e97da38-64be-43e0-802e-ac9232e18bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:09.644452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1083/2152] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:30:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:09.660004 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f298f2-a048-4cb9-914f-b534a176c877 service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:09.661491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-28f298f2-a048-4cb9-914f-b534a176c877 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:09.749144 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f298f2-a048-4cb9-914f-b534a176c877 service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:09.754062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628a7d2-b323-4f8f-92a4-89070ae88b07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:09.754661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628a7d2-b323-4f8f-92a4-89070ae88b07 service swift] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:09.754938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628a7d2-b323-4f8f-92a4-89070ae88b07 service swift] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:09.755222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628a7d2-b323-4f8f-92a4-89070ae88b07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:09.764732 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628a7d2-b323-4f8f-92a4-89070ae88b07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:09.767448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628a7d2-b323-4f8f-92a4-89070ae88b07 service swift] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:09.775963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1070/2153] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:30:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:30:10.000881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97da38-64be-43e0-802e-ac9232e18bc3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:10.002697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2e97da38-64be-43e0-802e-ac9232e18bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:10.040644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:10.105102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97da38-64be-43e0-802e-ac9232e18bc3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:10.111531 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813eb84-94ea-4070-aac8-68b3d5f2f64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:10.112374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813eb84-94ea-4070-aac8-68b3d5f2f64f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:10.112499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813eb84-94ea-4070-aac8-68b3d5f2f64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:10.112778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813eb84-94ea-4070-aac8-68b3d5f2f64f admin admin] PATH_INFO: `/v3/users/ec0a2c1fdc7945f6ac260c8e8ae7ad4d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:10.114767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813eb84-94ea-4070-aac8-68b3d5f2f64f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec0a2c1fdc7945f6ac260c8e8ae7ad4d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:10.117243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813eb84-94ea-4070-aac8-68b3d5f2f64f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:10.194868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d813eb84-94ea-4070-aac8-68b3d5f2f64f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec0a2c1fdc7945f6ac260c8e8ae7ad4d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:10.209122 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d813eb84-94ea-4070-aac8-68b3d5f2f64f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec0a2c1fdc7945f6ac260c8e8ae7ad4d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:10.237171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d813eb84-94ea-4070-aac8-68b3d5f2f64f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec0a2c1fdc7945f6ac260c8e8ae7ad4d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:10.256539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d813eb84-94ea-4070-aac8-68b3d5f2f64f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec0a2c1fdc7945f6ac260c8e8ae7ad4d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:10.267615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d813eb84-94ea-4070-aac8-68b3d5f2f64f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec0a2c1fdc7945f6ac260c8e8ae7ad4d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:10.272897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1084/2154] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:09 2022] DELETE /identity/v3/users/ec0a2c1fdc7945f6ac260c8e8ae7ad4d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:10.278505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05dbba6-64f9-49f0-b27b-aa76a3db17ad None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:10.279653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05dbba6-64f9-49f0-b27b-aa76a3db17ad None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:10.280135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05dbba6-64f9-49f0-b27b-aa76a3db17ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:10.312679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b05dbba6-64f9-49f0-b27b-aa76a3db17ad None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:10.314376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b05dbba6-64f9-49f0-b27b-aa76a3db17ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:10.421948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1071/2155] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:10.432759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813eb84-94ea-4070-aac8-68b3d5f2f64f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:10.434346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d813eb84-94ea-4070-aac8-68b3d5f2f64f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:10.581089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813eb84-94ea-4070-aac8-68b3d5f2f64f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:10.593143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311b8d5-60cc-4702-bc04-83bbd811294c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:10.593989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311b8d5-60cc-4702-bc04-83bbd811294c service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:10.594452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311b8d5-60cc-4702-bc04-83bbd811294c service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:10.594645 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311b8d5-60cc-4702-bc04-83bbd811294c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:10.609162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311b8d5-60cc-4702-bc04-83bbd811294c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:10.610737 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311b8d5-60cc-4702-bc04-83bbd811294c service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:10.637996 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1085/2156] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:10.992360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05dbba6-64f9-49f0-b27b-aa76a3db17ad None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:10.995372 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b05dbba6-64f9-49f0-b27b-aa76a3db17ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:11.028845 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05dbba6-64f9-49f0-b27b-aa76a3db17ad None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:11.039963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea3d24-9480-493b-8233-d478f044694c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:11.041937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea3d24-9480-493b-8233-d478f044694c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:11.042555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea3d24-9480-493b-8233-d478f044694c admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:11.043328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea3d24-9480-493b-8233-d478f044694c admin admin] PATH_INFO: `/v3/projects/6fa4238c07c0476095fde3edb52432ef` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:11.045219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea3d24-9480-493b-8233-d478f044694c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fa4238c07c0476095fde3edb52432ef)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:11.048251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea3d24-9480-493b-8233-d478f044694c admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:11.081998 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:11.121236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-20ea3d24-9480-493b-8233-d478f044694c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fa4238c07c0476095fde3edb52432ef', 'request_id': 'req-20ea3d24-9480-493b-8233-d478f044694c'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:11.138166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-20ea3d24-9480-493b-8233-d478f044694c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fa4238c07c0476095fde3edb52432ef', 'request_id': 'req-20ea3d24-9480-493b-8233-d478f044694c'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:11.156204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-20ea3d24-9480-493b-8233-d478f044694c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fa4238c07c0476095fde3edb52432ef', 'request_id': 'req-20ea3d24-9480-493b-8233-d478f044694c'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:11.187066 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-20ea3d24-9480-493b-8233-d478f044694c admin admin] The token cache is being invalidate because project 6fa4238c07c0476095fde3edb52432ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:11.187558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-20ea3d24-9480-493b-8233-d478f044694c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:11.188545 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-20ea3d24-9480-493b-8233-d478f044694c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:11.191048 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1072/2157] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:10 2022] DELETE /identity/v3/projects/6fa4238c07c0476095fde3edb52432ef => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:11.196143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae685bab-fd2f-4d1e-abae-5c83a1e64a0b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:11.196631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae685bab-fd2f-4d1e-abae-5c83a1e64a0b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:11.197100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae685bab-fd2f-4d1e-abae-5c83a1e64a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:11.221853 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ae685bab-fd2f-4d1e-abae-5c83a1e64a0b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:11.224169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ae685bab-fd2f-4d1e-abae-5c83a1e64a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:11.275736 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:11.332245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1086/2158] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:11.341246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4359ad-ae1a-485d-bfdf-aec77714c326 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:11.342106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4359ad-ae1a-485d-bfdf-aec77714c326 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:11.342385 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4359ad-ae1a-485d-bfdf-aec77714c326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:11.370938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-9d4359ad-ae1a-485d-bfdf-aec77714c326 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:11.372497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9d4359ad-ae1a-485d-bfdf-aec77714c326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:11.384685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1073/2159] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:11.389487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae685bab-fd2f-4d1e-abae-5c83a1e64a0b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:11.402769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae685bab-fd2f-4d1e-abae-5c83a1e64a0b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:11.410281 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07f411-5fe9-4851-991c-d7a114210520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:11.410934 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07f411-5fe9-4851-991c-d7a114210520 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:11.411603 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07f411-5fe9-4851-991c-d7a114210520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:11.411603 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07f411-5fe9-4851-991c-d7a114210520 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:11.414668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07f411-5fe9-4851-991c-d7a114210520 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:30:11.414988 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07f411-5fe9-4851-991c-d7a114210520 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:11.416752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07f411-5fe9-4851-991c-d7a114210520 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:11.440684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1087/2160] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:30:11.446843 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4359ad-ae1a-485d-bfdf-aec77714c326 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:11.474064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4359ad-ae1a-485d-bfdf-aec77714c326 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:11.490251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db576dbe-2d36-4e88-b4b7-dbfcef10949e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:11.491456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db576dbe-2d36-4e88-b4b7-dbfcef10949e admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:11.492266 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db576dbe-2d36-4e88-b4b7-dbfcef10949e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:11.493425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db576dbe-2d36-4e88-b4b7-dbfcef10949e admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:11.494783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db576dbe-2d36-4e88-b4b7-dbfcef10949e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:11.500109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db576dbe-2d36-4e88-b4b7-dbfcef10949e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:11.613435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1074/2161] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:30:11 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:30:11.616985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07f411-5fe9-4851-991c-d7a114210520 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:11.632944 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07f411-5fe9-4851-991c-d7a114210520 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:11.697146 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db576dbe-2d36-4e88-b4b7-dbfcef10949e admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:11.698859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-db576dbe-2d36-4e88-b4b7-dbfcef10949e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:11.781517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f8846-c6f0-42f8-ae72-764b2333bf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:11.782391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f8846-c6f0-42f8-ae72-764b2333bf9e admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:11.782678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f8846-c6f0-42f8-ae72-764b2333bf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:11.783004 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f8846-c6f0-42f8-ae72-764b2333bf9e admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:11.783720 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f8846-c6f0-42f8-ae72-764b2333bf9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:11.786193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f8846-c6f0-42f8-ae72-764b2333bf9e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:11.819528 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1088/2162] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:30:11 2022] POST /identity/v3/users => generated 290 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:30:11.824924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f8846-c6f0-42f8-ae72-764b2333bf9e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:11.837911 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db576dbe-2d36-4e88-b4b7-dbfcef10949e admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:11.838890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f8846-c6f0-42f8-ae72-764b2333bf9e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:11.846030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6f2c7-159e-4c0a-a7c4-9f83f2c80e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:11.847039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6f2c7-159e-4c0a-a7c4-9f83f2c80e1c admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:11.855118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6f2c7-159e-4c0a-a7c4-9f83f2c80e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:11.855245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34354f0d-5c5b-4a50-8327-61ee04beabe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:11.855366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6f2c7-159e-4c0a-a7c4-9f83f2c80e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:11.855466 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34354f0d-5c5b-4a50-8327-61ee04beabe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:11.855466 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34354f0d-5c5b-4a50-8327-61ee04beabe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:11.855670 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6f2c7-159e-4c0a-a7c4-9f83f2c80e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:11.855775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34354f0d-5c5b-4a50-8327-61ee04beabe8 admin admin] PATH_INFO: `/v3/users/2c2809d11db846908573b871b4200b77` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:11.855775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34354f0d-5c5b-4a50-8327-61ee04beabe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c2809d11db846908573b871b4200b77)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:11.855967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6f2c7-159e-4c0a-a7c4-9f83f2c80e1c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:11.856060 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34354f0d-5c5b-4a50-8327-61ee04beabe8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:11.870002 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1089/2163] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:11.876498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6f2c7-159e-4c0a-a7c4-9f83f2c80e1c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:11.889987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6f2c7-159e-4c0a-a7c4-9f83f2c80e1c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:11.896959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f06d71-d4c8-4647-a0a4-2e2def326681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:11.898005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f06d71-d4c8-4647-a0a4-2e2def326681 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:11.898595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f06d71-d4c8-4647-a0a4-2e2def326681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:11.899134 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f06d71-d4c8-4647-a0a4-2e2def326681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:11.900085 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f06d71-d4c8-4647-a0a4-2e2def326681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:11.902461 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f06d71-d4c8-4647-a0a4-2e2def326681 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:11.921657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1090/2164] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:11.925972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f06d71-d4c8-4647-a0a4-2e2def326681 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:11.947933 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-34354f0d-5c5b-4a50-8327-61ee04beabe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c2809d11db846908573b871b4200b77'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:11.971970 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-34354f0d-5c5b-4a50-8327-61ee04beabe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c2809d11db846908573b871b4200b77'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:11.977771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-34354f0d-5c5b-4a50-8327-61ee04beabe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c2809d11db846908573b871b4200b77'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:11.978531 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f06d71-d4c8-4647-a0a4-2e2def326681 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:11.978672 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-34354f0d-5c5b-4a50-8327-61ee04beabe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c2809d11db846908573b871b4200b77'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:11.987981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-34354f0d-5c5b-4a50-8327-61ee04beabe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c2809d11db846908573b871b4200b77'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:11.989475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed63b1-10e0-413c-a51f-8d19e6f41d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:11.989661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:11.990525 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed63b1-10e0-413c-a51f-8d19e6f41d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:11.990977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed63b1-10e0-413c-a51f-8d19e6f41d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:11.991398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed63b1-10e0-413c-a51f-8d19e6f41d60 admin admin] PATH_INFO: `/v3/projects/eaa5d21fc40540e4b811713e5efeff46/users/2d62232282dc4455834857d5290eb169/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:11.998828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1075/2165] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:11 2022] DELETE /identity/v3/users/2c2809d11db846908573b871b4200b77 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:12.004827 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452c78a-cb07-4b61-9185-e2253250fa87 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:12.005188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452c78a-cb07-4b61-9185-e2253250fa87 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:12.005805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452c78a-cb07-4b61-9185-e2253250fa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:12.021651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed63b1-10e0-413c-a51f-8d19e6f41d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa5d21fc40540e4b811713e5efeff46, user_id=2d62232282dc4455834857d5290eb169, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:12.025898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed63b1-10e0-413c-a51f-8d19e6f41d60 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:12.039196 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-7452c78a-cb07-4b61-9185-e2253250fa87 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:12.045329 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7452c78a-cb07-4b61-9185-e2253250fa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:12.055852 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1091/2166] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:11 2022] PUT /identity/v3/projects/eaa5d21fc40540e4b811713e5efeff46/users/2d62232282dc4455834857d5290eb169/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:12.059603 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25caedea-8e6f-4989-8479-257e3dcd97e8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:12.059990 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25caedea-8e6f-4989-8479-257e3dcd97e8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:12.060499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25caedea-8e6f-4989-8479-257e3dcd97e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:12.114777 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-25caedea-8e6f-4989-8479-257e3dcd97e8 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:12.120702 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-25caedea-8e6f-4989-8479-257e3dcd97e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:12.207227 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1092/2167] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:12.208222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25caedea-8e6f-4989-8479-257e3dcd97e8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:12.218635 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-25caedea-8e6f-4989-8479-257e3dcd97e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:12.218799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1076/2168] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:12.219614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452c78a-cb07-4b61-9185-e2253250fa87 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:12.220970 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7452c78a-cb07-4b61-9185-e2253250fa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:12.241608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25caedea-8e6f-4989-8479-257e3dcd97e8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:12.249552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab25599-8084-4018-b434-b057a62dedf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:12.250674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab25599-8084-4018-b434-b057a62dedf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:12.251017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab25599-8084-4018-b434-b057a62dedf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:12.251346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab25599-8084-4018-b434-b057a62dedf0 admin admin] PATH_INFO: `/v3/users/5a6718b4da5a4133b1d1a930c97ec560` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:12.253642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab25599-8084-4018-b434-b057a62dedf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a6718b4da5a4133b1d1a930c97ec560)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:12.256752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab25599-8084-4018-b434-b057a62dedf0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:12.274504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:12.321362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452c78a-cb07-4b61-9185-e2253250fa87 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:12.326911 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a0947-97ca-476a-a547-067281e658a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:12.327766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a0947-97ca-476a-a547-067281e658a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:12.328145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a0947-97ca-476a-a547-067281e658a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:12.328515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a0947-97ca-476a-a547-067281e658a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:12.360661 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8ab25599-8084-4018-b434-b057a62dedf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a6718b4da5a4133b1d1a930c97ec560'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:12.361465 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a0947-97ca-476a-a547-067281e658a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:12.363391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a0947-97ca-476a-a547-067281e658a3 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:12.375694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1077/2169] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:12.377116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a0947-97ca-476a-a547-067281e658a3 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:12.389224 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a0947-97ca-476a-a547-067281e658a3 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:12.392973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8ab25599-8084-4018-b434-b057a62dedf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a6718b4da5a4133b1d1a930c97ec560'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:12.395997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f5c60-6096-457b-87e5-53f8eb1747b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:12.396983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5c60-6096-457b-87e5-53f8eb1747b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:12.398003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5c60-6096-457b-87e5-53f8eb1747b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:12.399613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5c60-6096-457b-87e5-53f8eb1747b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:12.421828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f5c60-6096-457b-87e5-53f8eb1747b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:12.424763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8ab25599-8084-4018-b434-b057a62dedf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a6718b4da5a4133b1d1a930c97ec560'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:12.428231 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f5c60-6096-457b-87e5-53f8eb1747b5 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:12.434304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8ab25599-8084-4018-b434-b057a62dedf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a6718b4da5a4133b1d1a930c97ec560'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:12.435630 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8ab25599-8084-4018-b434-b057a62dedf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a6718b4da5a4133b1d1a930c97ec560'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:12.446771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1093/2170] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:12 2022] DELETE /identity/v3/users/5a6718b4da5a4133b1d1a930c97ec560 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:12.453408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696ace4-1f49-49f9-9bb6-feb8af46805c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:12.453877 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696ace4-1f49-49f9-9bb6-feb8af46805c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:12.454212 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696ace4-1f49-49f9-9bb6-feb8af46805c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:12.461099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1078/2171] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:12.478139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4696ace4-1f49-49f9-9bb6-feb8af46805c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:12.479427 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4696ace4-1f49-49f9-9bb6-feb8af46805c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:12.564128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1094/2172] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:30:12.570754 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f5c60-6096-457b-87e5-53f8eb1747b5 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:12.572307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-424f5c60-6096-457b-87e5-53f8eb1747b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:12.584536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f5c60-6096-457b-87e5-53f8eb1747b5 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:12.666880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2c606-01cc-4f97-9429-1ef6d48b1dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:12.669123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2c606-01cc-4f97-9429-1ef6d48b1dca service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:12.670040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2c606-01cc-4f97-9429-1ef6d48b1dca service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:12.671003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2c606-01cc-4f97-9429-1ef6d48b1dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:12.711141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2c606-01cc-4f97-9429-1ef6d48b1dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:12.712651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2c606-01cc-4f97-9429-1ef6d48b1dca service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:12.724802 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1079/2173] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:13.094066 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696ace4-1f49-49f9-9bb6-feb8af46805c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:13.096069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4696ace4-1f49-49f9-9bb6-feb8af46805c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:13.131507 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696ace4-1f49-49f9-9bb6-feb8af46805c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:13.156037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:13.157369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:13.157714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:13.158110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] PATH_INFO: `/v3/projects/b0032c2dc74d42f6894f198c0918e0b9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:13.160485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0032c2dc74d42f6894f198c0918e0b9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:13.162643 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:13.297342 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0032c2dc74d42f6894f198c0918e0b9', 'request_id': 'req-5b3c677c-1e42-4271-8948-58ac5887e75f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:13.308391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0032c2dc74d42f6894f198c0918e0b9', 'request_id': 'req-5b3c677c-1e42-4271-8948-58ac5887e75f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:13.325864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0032c2dc74d42f6894f198c0918e0b9', 'request_id': 'req-5b3c677c-1e42-4271-8948-58ac5887e75f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:13.344426 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] The token cache is being invalidate because project b0032c2dc74d42f6894f198c0918e0b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:13.344730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:13.356163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:13.360594 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1095/2174] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:13 2022] DELETE /identity/v3/projects/b0032c2dc74d42f6894f198c0918e0b9 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:30:13.366093 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebfa6d-751c-4df2-a624-ef557eb7ede6 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:13.366543 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebfa6d-751c-4df2-a624-ef557eb7ede6 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:13.366956 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebfa6d-751c-4df2-a624-ef557eb7ede6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:13.472756 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-beebfa6d-751c-4df2-a624-ef557eb7ede6 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:13.474407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-beebfa6d-751c-4df2-a624-ef557eb7ede6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:13.480771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:13.483366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:13.522208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:13.562378 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:13.633978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c677c-1e42-4271-8948-58ac5887e75f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:13.647107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a697f-231b-4557-bddc-68ec32178a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:13.648127 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a697f-231b-4557-bddc-68ec32178a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:13.648589 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a697f-231b-4557-bddc-68ec32178a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:13.649371 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a697f-231b-4557-bddc-68ec32178a2a admin admin] PATH_INFO: `/v3/projects/882b096f880e450d843e7c94dc63b26c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:13.651052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a697f-231b-4557-bddc-68ec32178a2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=882b096f880e450d843e7c94dc63b26c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:13.653631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a697f-231b-4557-bddc-68ec32178a2a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:13.671220 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1080/2175] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:13.684962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebfa6d-751c-4df2-a624-ef557eb7ede6 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:13.688412 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-beebfa6d-751c-4df2-a624-ef557eb7ede6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:13.731747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0c6a697f-231b-4557-bddc-68ec32178a2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '882b096f880e450d843e7c94dc63b26c', 'request_id': 'req-0c6a697f-231b-4557-bddc-68ec32178a2a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:13.732727 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebfa6d-751c-4df2-a624-ef557eb7ede6 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:13.739198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0c6a697f-231b-4557-bddc-68ec32178a2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '882b096f880e450d843e7c94dc63b26c', 'request_id': 'req-0c6a697f-231b-4557-bddc-68ec32178a2a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:13.750899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0c6a697f-231b-4557-bddc-68ec32178a2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '882b096f880e450d843e7c94dc63b26c', 'request_id': 'req-0c6a697f-231b-4557-bddc-68ec32178a2a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:13.790564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0c6a697f-231b-4557-bddc-68ec32178a2a admin admin] The token cache is being invalidate because project 882b096f880e450d843e7c94dc63b26c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:13.792829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0c6a697f-231b-4557-bddc-68ec32178a2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:13.794452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0c6a697f-231b-4557-bddc-68ec32178a2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:13.803259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1096/2176] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:13 2022] DELETE /identity/v3/projects/882b096f880e450d843e7c94dc63b26c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:30:13.804678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830e571-6d81-4967-886d-9bd5d67354d4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:13.804963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830e571-6d81-4967-886d-9bd5d67354d4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:13.805241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830e571-6d81-4967-886d-9bd5d67354d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:13.857907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a830e571-6d81-4967-886d-9bd5d67354d4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:13.859288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a830e571-6d81-4967-886d-9bd5d67354d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:13.905393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7d98f-b05e-42dc-b06c-ff67800a9182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:13.906117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7d98f-b05e-42dc-b06c-ff67800a9182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:13.906668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7d98f-b05e-42dc-b06c-ff67800a9182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:13.906945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7d98f-b05e-42dc-b06c-ff67800a9182 admin admin] PATH_INFO: `/v3/users/41d457c7846340c79e35b441a8ec4cbd` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:13.908157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7d98f-b05e-42dc-b06c-ff67800a9182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d457c7846340c79e35b441a8ec4cbd)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:13.910245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7d98f-b05e-42dc-b06c-ff67800a9182 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:13.941828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1097/2177] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:13.943786 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596a01d-2ff4-446e-824f-5e8c49e350d9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:13.944075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596a01d-2ff4-446e-824f-5e8c49e350d9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:13.944355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596a01d-2ff4-446e-824f-5e8c49e350d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:13.973389 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c596a01d-2ff4-446e-824f-5e8c49e350d9 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:13.974644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c596a01d-2ff4-446e-824f-5e8c49e350d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:13.987056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1098/2178] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:13.988346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596a01d-2ff4-446e-824f-5e8c49e350d9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:14.007726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596a01d-2ff4-446e-824f-5e8c49e350d9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:14.013621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e081d62-d747-452c-b79e-720ae2559cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:14.014714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-25f7d98f-b05e-42dc-b06c-ff67800a9182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d457c7846340c79e35b441a8ec4cbd'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:14.016049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e081d62-d747-452c-b79e-720ae2559cdb admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.016547 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e081d62-d747-452c-b79e-720ae2559cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.016837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e081d62-d747-452c-b79e-720ae2559cdb admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:14.017512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e081d62-d747-452c-b79e-720ae2559cdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:30:14.017862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e081d62-d747-452c-b79e-720ae2559cdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:14.020920 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e081d62-d747-452c-b79e-720ae2559cdb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:14.040816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1099/2179] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:30:14.042637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22e659-25e4-40e6-b39f-1d5c709ba19e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.042978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22e659-25e4-40e6-b39f-1d5c709ba19e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.043325 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22e659-25e4-40e6-b39f-1d5c709ba19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:14.051310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-25f7d98f-b05e-42dc-b06c-ff67800a9182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d457c7846340c79e35b441a8ec4cbd'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:14.072829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-de22e659-25e4-40e6-b39f-1d5c709ba19e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:14.074290 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-25f7d98f-b05e-42dc-b06c-ff67800a9182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d457c7846340c79e35b441a8ec4cbd'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:14.074813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-de22e659-25e4-40e6-b39f-1d5c709ba19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:14.075693 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-25f7d98f-b05e-42dc-b06c-ff67800a9182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d457c7846340c79e35b441a8ec4cbd'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:14.090151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-25f7d98f-b05e-42dc-b06c-ff67800a9182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d457c7846340c79e35b441a8ec4cbd'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:14.098119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1100/2180] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:14.098976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22e659-25e4-40e6-b39f-1d5c709ba19e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:14.111616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1081/2181] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:13 2022] DELETE /identity/v3/users/41d457c7846340c79e35b441a8ec4cbd => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:14.116209 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7d98f-b05e-42dc-b06c-ff67800a9182 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:14.118738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-25f7d98f-b05e-42dc-b06c-ff67800a9182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:14.125515 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22e659-25e4-40e6-b39f-1d5c709ba19e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:14.157126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:14.198870 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:14.236145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7d98f-b05e-42dc-b06c-ff67800a9182 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:14.243950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043537d9-1949-4ea9-924c-71c8c4cdcae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:14.244584 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043537d9-1949-4ea9-924c-71c8c4cdcae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.244864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043537d9-1949-4ea9-924c-71c8c4cdcae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.245142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043537d9-1949-4ea9-924c-71c8c4cdcae6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:14.245791 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043537d9-1949-4ea9-924c-71c8c4cdcae6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:30:14.246095 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043537d9-1949-4ea9-924c-71c8c4cdcae6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:14.250446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043537d9-1949-4ea9-924c-71c8c4cdcae6 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:14.268669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1082/2182] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:30:14.268669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44e644-d27f-4aeb-bdc4-e4ec096614ca None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.268669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44e644-d27f-4aeb-bdc4-e4ec096614ca None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.268669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44e644-d27f-4aeb-bdc4-e4ec096614ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:14.284686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a0acc-2511-4918-9757-725034032d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:14.286731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a0acc-2511-4918-9757-725034032d85 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.289911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a0acc-2511-4918-9757-725034032d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.291000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a0acc-2511-4918-9757-725034032d85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:14.292166 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a0acc-2511-4918-9757-725034032d85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:14.295316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-3f44e644-d27f-4aeb-bdc4-e4ec096614ca None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:14.295864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a0acc-2511-4918-9757-725034032d85 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:14.296803 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3f44e644-d27f-4aeb-bdc4-e4ec096614ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:14.306030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1083/2183] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:14.306991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44e644-d27f-4aeb-bdc4-e4ec096614ca None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:14.320535 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44e644-d27f-4aeb-bdc4-e4ec096614ca None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:14.331983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15624294-5e00-4d27-aa3e-f88e71b9576c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:14.332783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15624294-5e00-4d27-aa3e-f88e71b9576c admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.333653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15624294-5e00-4d27-aa3e-f88e71b9576c admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.335675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15624294-5e00-4d27-aa3e-f88e71b9576c admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:14.336751 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15624294-5e00-4d27-aa3e-f88e71b9576c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:14.339762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15624294-5e00-4d27-aa3e-f88e71b9576c admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:14.362062 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1101/2184] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:30:14 2022] POST /identity/v3/projects => generated 399 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:14.366608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a0acc-2511-4918-9757-725034032d85 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:14.370052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9d9a0acc-2511-4918-9757-725034032d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:14.388559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1084/2185] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:30:14 2022] POST /identity/v3/projects => generated 389 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:14.390640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15624294-5e00-4d27-aa3e-f88e71b9576c admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:14.396188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-15624294-5e00-4d27-aa3e-f88e71b9576c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:14.528491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a0acc-2511-4918-9757-725034032d85 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:14.564344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15624294-5e00-4d27-aa3e-f88e71b9576c admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:14.583774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866612b-7c76-4432-9fa6-a7e0ecb30c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:14.584479 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866612b-7c76-4432-9fa6-a7e0ecb30c33 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.584820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866612b-7c76-4432-9fa6-a7e0ecb30c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.585945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866612b-7c76-4432-9fa6-a7e0ecb30c33 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:14.587251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0866612b-7c76-4432-9fa6-a7e0ecb30c33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:14.594799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0866612b-7c76-4432-9fa6-a7e0ecb30c33 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:14.662685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1085/2186] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:30:14 2022] POST /identity/v3/users => generated 300 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:14.662685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866612b-7c76-4432-9fa6-a7e0ecb30c33 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:14.681537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b651d7f-b5ba-4106-937c-b7d296e546da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:14.682881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866612b-7c76-4432-9fa6-a7e0ecb30c33 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:14.683247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b651d7f-b5ba-4106-937c-b7d296e546da service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.683741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b651d7f-b5ba-4106-937c-b7d296e546da service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.684260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b651d7f-b5ba-4106-937c-b7d296e546da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:14.700035 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec6f62-77e6-4091-af49-f617b53d5e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:14.700734 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec6f62-77e6-4091-af49-f617b53d5e53 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.701021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec6f62-77e6-4091-af49-f617b53d5e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.701348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec6f62-77e6-4091-af49-f617b53d5e53 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:14.702035 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec6f62-77e6-4091-af49-f617b53d5e53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:14.704648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec6f62-77e6-4091-af49-f617b53d5e53 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:14.726647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b651d7f-b5ba-4106-937c-b7d296e546da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:14.730581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b651d7f-b5ba-4106-937c-b7d296e546da service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:14.733862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1086/2187] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:30:14 2022] POST /identity/v3/users => generated 295 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:14.735521 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec6f62-77e6-4091-af49-f617b53d5e53 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:14.754836 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec6f62-77e6-4091-af49-f617b53d5e53 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:14.761289 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1102/2188] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:14.762831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b651d7f-b5ba-4106-937c-b7d296e546da service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:14.774102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b090f-db1d-4939-a7c0-e19f325dcade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:14.775635 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b090f-db1d-4939-a7c0-e19f325dcade admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.775927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b090f-db1d-4939-a7c0-e19f325dcade admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.776204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b090f-db1d-4939-a7c0-e19f325dcade admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:14.778865 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b090f-db1d-4939-a7c0-e19f325dcade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:14.784832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b651d7f-b5ba-4106-937c-b7d296e546da service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:14.785127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b090f-db1d-4939-a7c0-e19f325dcade admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:14.802275 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c195b0d-29f0-41ec-b99f-1576f509e41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:14.803131 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195b0d-29f0-41ec-b99f-1576f509e41d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.805798 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195b0d-29f0-41ec-b99f-1576f509e41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.806112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195b0d-29f0-41ec-b99f-1576f509e41d admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:14.806866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c195b0d-29f0-41ec-b99f-1576f509e41d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:14.813074 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1087/2189] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:14.813245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c195b0d-29f0-41ec-b99f-1576f509e41d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:14.814841 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b090f-db1d-4939-a7c0-e19f325dcade admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:14.842125 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1103/2190] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:14.843700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c195b0d-29f0-41ec-b99f-1576f509e41d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:14.848285 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b090f-db1d-4939-a7c0-e19f325dcade admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:14.861702 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5dfa7-a171-4bb5-9ff5-024b9725e7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:14.865359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5dfa7-a171-4bb5-9ff5-024b9725e7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.866873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5dfa7-a171-4bb5-9ff5-024b9725e7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.867150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5dfa7-a171-4bb5-9ff5-024b9725e7c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:14.869406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c195b0d-29f0-41ec-b99f-1576f509e41d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:14.870153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5dfa7-a171-4bb5-9ff5-024b9725e7c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:14.876077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5dfa7-a171-4bb5-9ff5-024b9725e7c9 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:14.892272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683f242-b58f-47fa-b57a-ece0d38c4e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:14.892272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683f242-b58f-47fa-b57a-ece0d38c4e22 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.892272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683f242-b58f-47fa-b57a-ece0d38c4e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.892272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683f242-b58f-47fa-b57a-ece0d38c4e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:14.892835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683f242-b58f-47fa-b57a-ece0d38c4e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:14.896601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683f242-b58f-47fa-b57a-ece0d38c4e22 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:14.921996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1088/2191] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:14.922640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1104/2192] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:30:14.924359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5dfa7-a171-4bb5-9ff5-024b9725e7c9 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:14.925075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683f242-b58f-47fa-b57a-ece0d38c4e22 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:14.953936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683f242-b58f-47fa-b57a-ece0d38c4e22 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:14.954437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5dfa7-a171-4bb5-9ff5-024b9725e7c9 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:14.961199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084fa56-0e07-442e-8200-fbbb3390c58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:14.962247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084fa56-0e07-442e-8200-fbbb3390c58d admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.962955 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084fa56-0e07-442e-8200-fbbb3390c58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.963839 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084fa56-0e07-442e-8200-fbbb3390c58d admin admin] PATH_INFO: `/v3/projects/d6e9ad6b0fd043be84d8f82f276e913f/users/cd1582f15be0450ca1e461251a2b515c/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:14.976578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2ad9f-7b11-4568-a7a3-20bd3e751bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:14.981216 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2ad9f-7b11-4568-a7a3-20bd3e751bb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:14.981657 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2ad9f-7b11-4568-a7a3-20bd3e751bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:14.986589 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2ad9f-7b11-4568-a7a3-20bd3e751bb8 admin admin] PATH_INFO: `/v3/projects/27a70cfc525344cc97780ba1238e350a/users/b14c600226a1444fbaa0e82971f40deb/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:15.021596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084fa56-0e07-442e-8200-fbbb3390c58d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e9ad6b0fd043be84d8f82f276e913f, user_id=cd1582f15be0450ca1e461251a2b515c, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:15.026053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084fa56-0e07-442e-8200-fbbb3390c58d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:15.039337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2ad9f-7b11-4568-a7a3-20bd3e751bb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27a70cfc525344cc97780ba1238e350a, user_id=b14c600226a1444fbaa0e82971f40deb, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:15.053904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2ad9f-7b11-4568-a7a3-20bd3e751bb8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:15.063802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1105/2193] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:14 2022] PUT /identity/v3/projects/d6e9ad6b0fd043be84d8f82f276e913f/users/cd1582f15be0450ca1e461251a2b515c/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:15.082631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ef990-136e-4938-b9d4-48b81366cc03 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:15.083472 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ef990-136e-4938-b9d4-48b81366cc03 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:15.083623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ef990-136e-4938-b9d4-48b81366cc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:15.101177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1089/2194] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:14 2022] PUT /identity/v3/projects/27a70cfc525344cc97780ba1238e350a/users/b14c600226a1444fbaa0e82971f40deb/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:15.115203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd56b7-8fc2-4bd8-94ae-e03123f2f7e6 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:15.115600 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd56b7-8fc2-4bd8-94ae-e03123f2f7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:15.115940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd56b7-8fc2-4bd8-94ae-e03123f2f7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:15.160754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:15.168827 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-144ef990-136e-4938-b9d4-48b81366cc03 None None] MFA Rules not processed for user `cd1582f15be0450ca1e461251a2b515c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:15.170729 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-144ef990-136e-4938-b9d4-48b81366cc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:15.179445 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-33dd56b7-8fc2-4bd8-94ae-e03123f2f7e6 None None] MFA Rules not processed for user `b14c600226a1444fbaa0e82971f40deb`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:15.184785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-33dd56b7-8fc2-4bd8-94ae-e03123f2f7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:15.293859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:15.394384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1106/2195] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:15.408686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ef990-136e-4938-b9d4-48b81366cc03 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:15.409428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1090/2196] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:15.412148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-144ef990-136e-4938-b9d4-48b81366cc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:15.421606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd56b7-8fc2-4bd8-94ae-e03123f2f7e6 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:15.424891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-33dd56b7-8fc2-4bd8-94ae-e03123f2f7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:15.617365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd56b7-8fc2-4bd8-94ae-e03123f2f7e6 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:15.625017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c030e-64ba-4d93-a440-e6c9fefad7bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:15.625309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ef990-136e-4938-b9d4-48b81366cc03 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:15.625649 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c030e-64ba-4d93-a440-e6c9fefad7bf service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:15.625930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c030e-64ba-4d93-a440-e6c9fefad7bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:15.626207 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c030e-64ba-4d93-a440-e6c9fefad7bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:15.631490 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56726a-3188-4e40-bd99-4c1bb70d03ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:15.633785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56726a-3188-4e40-bd99-4c1bb70d03ba service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:15.634258 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56726a-3188-4e40-bd99-4c1bb70d03ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:15.634638 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56726a-3188-4e40-bd99-4c1bb70d03ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:15.644813 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c030e-64ba-4d93-a440-e6c9fefad7bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:15.646311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c030e-64ba-4d93-a440-e6c9fefad7bf service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:15.650119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56726a-3188-4e40-bd99-4c1bb70d03ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:15.652687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56726a-3188-4e40-bd99-4c1bb70d03ba service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:15.673965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1091/2197] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:15.693378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1107/2198] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:15.875907 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c030e-64ba-4d93-a440-e6c9fefad7bf service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:15.884931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a26c030e-64ba-4d93-a440-e6c9fefad7bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:16.070315 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c030e-64ba-4d93-a440-e6c9fefad7bf service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:16.078536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:16.079473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:16.080285 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:16.081059 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] PATH_INFO: `/v3/projects/d1d87447cbcc42779b39aaaa511c72e5` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:16.090328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d87447cbcc42779b39aaaa511c72e5)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:16.094475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:16.208164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d87447cbcc42779b39aaaa511c72e5', 'request_id': 'req-677c68e3-3b9e-4c19-968f-673afd562a95'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:16.219032 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d87447cbcc42779b39aaaa511c72e5', 'request_id': 'req-677c68e3-3b9e-4c19-968f-673afd562a95'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:16.234307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d87447cbcc42779b39aaaa511c72e5', 'request_id': 'req-677c68e3-3b9e-4c19-968f-673afd562a95'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:16.281123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] The token cache is being invalidate because project d1d87447cbcc42779b39aaaa511c72e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:16.283324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:16.288033 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:16.296158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56726a-3188-4e40-bd99-4c1bb70d03ba service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:16.320952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1092/2199] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:15 2022] DELETE /identity/v3/projects/d1d87447cbcc42779b39aaaa511c72e5 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:16.338201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56726a-3188-4e40-bd99-4c1bb70d03ba service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:16.701557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c53e7c-ee69-4244-b33e-ca8d2ea32c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:16.704827 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c53e7c-ee69-4244-b33e-ca8d2ea32c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:16.705268 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c53e7c-ee69-4244-b33e-ca8d2ea32c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:16.708402 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c53e7c-ee69-4244-b33e-ca8d2ea32c16 admin admin] PATH_INFO: `/v3/users/41e85af97a8249f1b81f263deff04f6e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:16.711267 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c53e7c-ee69-4244-b33e-ca8d2ea32c16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41e85af97a8249f1b81f263deff04f6e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:16.713691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c53e7c-ee69-4244-b33e-ca8d2ea32c16 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:16.939217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89c53e7c-ee69-4244-b33e-ca8d2ea32c16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41e85af97a8249f1b81f263deff04f6e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:16.941987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:16.973732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89c53e7c-ee69-4244-b33e-ca8d2ea32c16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41e85af97a8249f1b81f263deff04f6e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:16.989836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89c53e7c-ee69-4244-b33e-ca8d2ea32c16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41e85af97a8249f1b81f263deff04f6e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:17.002276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89c53e7c-ee69-4244-b33e-ca8d2ea32c16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41e85af97a8249f1b81f263deff04f6e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:17.003758 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-89c53e7c-ee69-4244-b33e-ca8d2ea32c16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41e85af97a8249f1b81f263deff04f6e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:17.021037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1108/2200] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:16 2022] DELETE /identity/v3/users/41e85af97a8249f1b81f263deff04f6e => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:18.002203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:18.004557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:18.040620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:18.046053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c68e3-3b9e-4c19-968f-673afd562a95 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:18.133135 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cbd46-b96b-44a5-882b-966819e108d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:18.133783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cbd46-b96b-44a5-882b-966819e108d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:18.134069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cbd46-b96b-44a5-882b-966819e108d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:18.134347 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cbd46-b96b-44a5-882b-966819e108d2 admin admin] PATH_INFO: `/v3/projects/8a37123d72eb402f96aabc0f1aaa7239` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:18.135839 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cbd46-b96b-44a5-882b-966819e108d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a37123d72eb402f96aabc0f1aaa7239)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:18.138370 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cbd46-b96b-44a5-882b-966819e108d2 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:18.205915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5e1cbd46-b96b-44a5-882b-966819e108d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a37123d72eb402f96aabc0f1aaa7239', 'request_id': 'req-5e1cbd46-b96b-44a5-882b-966819e108d2'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:18.214259 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5e1cbd46-b96b-44a5-882b-966819e108d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a37123d72eb402f96aabc0f1aaa7239', 'request_id': 'req-5e1cbd46-b96b-44a5-882b-966819e108d2'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:18.236081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5e1cbd46-b96b-44a5-882b-966819e108d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a37123d72eb402f96aabc0f1aaa7239', 'request_id': 'req-5e1cbd46-b96b-44a5-882b-966819e108d2'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:18.250944 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5e1cbd46-b96b-44a5-882b-966819e108d2 admin admin] The token cache is being invalidate because project 8a37123d72eb402f96aabc0f1aaa7239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:18.251300 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5e1cbd46-b96b-44a5-882b-966819e108d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:18.252051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5e1cbd46-b96b-44a5-882b-966819e108d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:18.257510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1093/2201] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:17 2022] DELETE /identity/v3/projects/8a37123d72eb402f96aabc0f1aaa7239 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:18.258876 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d8513-a4e9-40d6-8db7-c1e0f2cfc827 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:18.259341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d8513-a4e9-40d6-8db7-c1e0f2cfc827 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:18.259436 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d8513-a4e9-40d6-8db7-c1e0f2cfc827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:18.333375 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-392d8513-a4e9-40d6-8db7-c1e0f2cfc827 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:18.339347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-392d8513-a4e9-40d6-8db7-c1e0f2cfc827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:18.493597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1109/2202] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:18.502373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9da57-8231-43b8-b6e1-d4f632c88e6c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:18.502581 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9da57-8231-43b8-b6e1-d4f632c88e6c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:18.503881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9da57-8231-43b8-b6e1-d4f632c88e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:18.557373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-89b9da57-8231-43b8-b6e1-d4f632c88e6c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:18.561649 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-89b9da57-8231-43b8-b6e1-d4f632c88e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:18.591602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1094/2203] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:18.596324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d8513-a4e9-40d6-8db7-c1e0f2cfc827 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:18.618762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d8513-a4e9-40d6-8db7-c1e0f2cfc827 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:18.627070 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f006ff-3652-4e20-98fc-3fda9fcc0215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:18.627722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f006ff-3652-4e20-98fc-3fda9fcc0215 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:18.628013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f006ff-3652-4e20-98fc-3fda9fcc0215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:18.628292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f006ff-3652-4e20-98fc-3fda9fcc0215 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:18.628962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f006ff-3652-4e20-98fc-3fda9fcc0215 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:30:18.629264 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f006ff-3652-4e20-98fc-3fda9fcc0215 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:18.630862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f006ff-3652-4e20-98fc-3fda9fcc0215 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:18.673163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9da57-8231-43b8-b6e1-d4f632c88e6c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:18.679766 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1110/2204] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:30:18.691630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9da57-8231-43b8-b6e1-d4f632c88e6c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:18.698384 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03198106-8a5c-4386-9e56-ab0fe214ddec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:18.699953 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03198106-8a5c-4386-9e56-ab0fe214ddec admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:18.700236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03198106-8a5c-4386-9e56-ab0fe214ddec admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:18.700555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03198106-8a5c-4386-9e56-ab0fe214ddec admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:18.701198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03198106-8a5c-4386-9e56-ab0fe214ddec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:18.703129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03198106-8a5c-4386-9e56-ab0fe214ddec admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:18.743679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1095/2205] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:30:18 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:18.748426 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f006ff-3652-4e20-98fc-3fda9fcc0215 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:18.761920 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f006ff-3652-4e20-98fc-3fda9fcc0215 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:18.846866 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4b54a-dbd8-4756-90e2-2eb07ca78d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:18.847603 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4b54a-dbd8-4756-90e2-2eb07ca78d9e admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:18.847987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4b54a-dbd8-4756-90e2-2eb07ca78d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:18.848332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4b54a-dbd8-4756-90e2-2eb07ca78d9e admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:18.849038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4b54a-dbd8-4756-90e2-2eb07ca78d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:18.867260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4b54a-dbd8-4756-90e2-2eb07ca78d9e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:18.903550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1111/2206] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:30:18 2022] POST /identity/v3/users => generated 291 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:18.905939 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03198106-8a5c-4386-9e56-ab0fe214ddec admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:18.926420 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03198106-8a5c-4386-9e56-ab0fe214ddec admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:18.933621 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5a04f-9aba-4d29-8dcd-ed18de521d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:18.934685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5a04f-9aba-4d29-8dcd-ed18de521d03 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:18.935981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5a04f-9aba-4d29-8dcd-ed18de521d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:18.936304 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5a04f-9aba-4d29-8dcd-ed18de521d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:18.937125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5a04f-9aba-4d29-8dcd-ed18de521d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:18.939191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5a04f-9aba-4d29-8dcd-ed18de521d03 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:18.953288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1096/2207] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:18.958695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4b54a-dbd8-4756-90e2-2eb07ca78d9e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:18.973527 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4b54a-dbd8-4756-90e2-2eb07ca78d9e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:18.994802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecaf80-fdeb-4050-9619-2d726d8d1d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:18.995797 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecaf80-fdeb-4050-9619-2d726d8d1d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:18.997279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecaf80-fdeb-4050-9619-2d726d8d1d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:18.998964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecaf80-fdeb-4050-9619-2d726d8d1d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:19.001497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecaf80-fdeb-4050-9619-2d726d8d1d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:19.012285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecaf80-fdeb-4050-9619-2d726d8d1d0b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:19.033198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5a04f-9aba-4d29-8dcd-ed18de521d03 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:19.033752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1112/2208] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:19.046529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5a04f-9aba-4d29-8dcd-ed18de521d03 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:19.053100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a083de0-a70f-44fe-8b6e-0613a63274ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:19.053770 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a083de0-a70f-44fe-8b6e-0613a63274ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:19.054712 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a083de0-a70f-44fe-8b6e-0613a63274ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:19.055071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a083de0-a70f-44fe-8b6e-0613a63274ae admin admin] PATH_INFO: `/v3/projects/9133ab50b25247bca270278c37632237/users/e248f668495943ba8eaa115d22349fbf/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:19.073644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a083de0-a70f-44fe-8b6e-0613a63274ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=9133ab50b25247bca270278c37632237, user_id=e248f668495943ba8eaa115d22349fbf, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:19.076728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a083de0-a70f-44fe-8b6e-0613a63274ae admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:19.102635 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1097/2209] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:19 2022] PUT /identity/v3/projects/9133ab50b25247bca270278c37632237/users/e248f668495943ba8eaa115d22349fbf/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:19.117128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f8675-db0c-4225-9005-782d41eab1fe None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:19.117520 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f8675-db0c-4225-9005-782d41eab1fe None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:19.117782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f8675-db0c-4225-9005-782d41eab1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:19.165460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ee3f8675-db0c-4225-9005-782d41eab1fe None None] MFA Rules not processed for user `e248f668495943ba8eaa115d22349fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:19.167071 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ee3f8675-db0c-4225-9005-782d41eab1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:19.203315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:19.285408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1113/2210] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:19.286626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a083de0-a70f-44fe-8b6e-0613a63274ae admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:19.388475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:19.449737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a083de0-a70f-44fe-8b6e-0613a63274ae admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:19.456425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ceb272-6ebc-41b5-8bb2-4243391d6cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:19.457085 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ceb272-6ebc-41b5-8bb2-4243391d6cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:19.457475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ceb272-6ebc-41b5-8bb2-4243391d6cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:19.457667 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ceb272-6ebc-41b5-8bb2-4243391d6cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:19.458374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ceb272-6ebc-41b5-8bb2-4243391d6cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:19.460406 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ceb272-6ebc-41b5-8bb2-4243391d6cb6 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:19.505192 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f8675-db0c-4225-9005-782d41eab1fe None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:19.507931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1098/2211] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:30:19 2022] POST /identity/v3/projects => generated 383 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:19.526911 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f8675-db0c-4225-9005-782d41eab1fe None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:19.653480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeff6ba-2265-4fea-8351-d9fffa90b001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:19.655058 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff6ba-2265-4fea-8351-d9fffa90b001 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:19.655892 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff6ba-2265-4fea-8351-d9fffa90b001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:19.655892 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff6ba-2265-4fea-8351-d9fffa90b001 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:19.656757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeff6ba-2265-4fea-8351-d9fffa90b001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:19.658874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeff6ba-2265-4fea-8351-d9fffa90b001 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:19.737274 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ceb272-6ebc-41b5-8bb2-4243391d6cb6 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:19.739776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1114/2212] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:30:19 2022] POST /identity/v3/users => generated 298 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:30:19.758368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ceb272-6ebc-41b5-8bb2-4243391d6cb6 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:19.772579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8a0f3-ca86-4cf9-b85d-511ede106dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:19.773230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8a0f3-ca86-4cf9-b85d-511ede106dce admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:19.773717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8a0f3-ca86-4cf9-b85d-511ede106dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:19.774068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8a0f3-ca86-4cf9-b85d-511ede106dce admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:19.775083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8a0f3-ca86-4cf9-b85d-511ede106dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:19.777504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8a0f3-ca86-4cf9-b85d-511ede106dce admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:19.809387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1099/2213] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:19.814541 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeff6ba-2265-4fea-8351-d9fffa90b001 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:19.831139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeff6ba-2265-4fea-8351-d9fffa90b001 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:19.846801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8287e1-410d-4084-9c23-a58ae2dd863b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:19.847499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8287e1-410d-4084-9c23-a58ae2dd863b admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:19.847816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8287e1-410d-4084-9c23-a58ae2dd863b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:19.849186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8287e1-410d-4084-9c23-a58ae2dd863b admin admin] PATH_INFO: `/v3/projects/02834a352fd8489b85f5016011529f3f/users/e3536ec4edfe4e1aa3dc77fdf7bf110f/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:19.898393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8287e1-410d-4084-9c23-a58ae2dd863b admin admin] RBAC: Authorizing `identity:create_grant(project_id=02834a352fd8489b85f5016011529f3f, user_id=e3536ec4edfe4e1aa3dc77fdf7bf110f, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:19.901272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8287e1-410d-4084-9c23-a58ae2dd863b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:19.942145 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1115/2214] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:19 2022] PUT /identity/v3/projects/02834a352fd8489b85f5016011529f3f/users/e3536ec4edfe4e1aa3dc77fdf7bf110f/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:19.946944 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8a0f3-ca86-4cf9-b85d-511ede106dce admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:19.975759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8a0f3-ca86-4cf9-b85d-511ede106dce admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:20.149531 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8adb88b-49d9-4e59-8fc4-cae29d91a928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:20.150320 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adb88b-49d9-4e59-8fc4-cae29d91a928 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:20.151877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adb88b-49d9-4e59-8fc4-cae29d91a928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:20.152989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adb88b-49d9-4e59-8fc4-cae29d91a928 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:20.155241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8adb88b-49d9-4e59-8fc4-cae29d91a928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:20.159311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8adb88b-49d9-4e59-8fc4-cae29d91a928 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:20.185270 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1100/2215] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:20.189339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8287e1-410d-4084-9c23-a58ae2dd863b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:20.202891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8287e1-410d-4084-9c23-a58ae2dd863b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:20.210090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7d382-06f4-41b7-bdc6-f7537deb5181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:20.210860 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7d382-06f4-41b7-bdc6-f7537deb5181 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:20.211206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7d382-06f4-41b7-bdc6-f7537deb5181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:20.211552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7d382-06f4-41b7-bdc6-f7537deb5181 admin admin] PATH_INFO: `/v3/projects/02834a352fd8489b85f5016011529f3f/users/e3536ec4edfe4e1aa3dc77fdf7bf110f/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:20.215198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c7d382-06f4-41b7-bdc6-f7537deb5181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02834a352fd8489b85f5016011529f3f, user_id=e3536ec4edfe4e1aa3dc77fdf7bf110f, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:20.225700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c7d382-06f4-41b7-bdc6-f7537deb5181 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:20.253774 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1116/2216] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:20 2022] PUT /identity/v3/projects/02834a352fd8489b85f5016011529f3f/users/e3536ec4edfe4e1aa3dc77fdf7bf110f/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:20.270465 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e2f06-ef61-40e2-a8d7-5d3ebe48f37c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:20.270971 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e2f06-ef61-40e2-a8d7-5d3ebe48f37c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:20.271314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e2f06-ef61-40e2-a8d7-5d3ebe48f37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:20.384357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-870e2f06-ef61-40e2-a8d7-5d3ebe48f37c None None] MFA Rules not processed for user `e3536ec4edfe4e1aa3dc77fdf7bf110f`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:20.389609 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-870e2f06-ef61-40e2-a8d7-5d3ebe48f37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:20.566522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1101/2217] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:20.585839 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7d382-06f4-41b7-bdc6-f7537deb5181 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:20.587972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b7c7d382-06f4-41b7-bdc6-f7537deb5181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:20.747872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7d382-06f4-41b7-bdc6-f7537deb5181 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:20.757350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c04557-e683-4feb-9fbd-d8def17ac166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:20.758249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c04557-e683-4feb-9fbd-d8def17ac166 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:20.758729 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c04557-e683-4feb-9fbd-d8def17ac166 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:20.759183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c04557-e683-4feb-9fbd-d8def17ac166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:20.771306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c04557-e683-4feb-9fbd-d8def17ac166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:20.773167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c04557-e683-4feb-9fbd-d8def17ac166 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:20.788247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1117/2218] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:20.943228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e2f06-ef61-40e2-a8d7-5d3ebe48f37c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:20.956509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e2f06-ef61-40e2-a8d7-5d3ebe48f37c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:20.961320 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309e944-1b73-4dfb-902a-b102c570bd50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:20.963626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309e944-1b73-4dfb-902a-b102c570bd50 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:20.964152 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309e944-1b73-4dfb-902a-b102c570bd50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:20.964605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309e944-1b73-4dfb-902a-b102c570bd50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:21.003130 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:21.066752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309e944-1b73-4dfb-902a-b102c570bd50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:21.069073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309e944-1b73-4dfb-902a-b102c570bd50 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:21.092683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1102/2219] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:22.348205 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c4e60-0292-4a12-bc77-519cffec1c0e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:22.349274 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c4e60-0292-4a12-bc77-519cffec1c0e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:22.349805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c4e60-0292-4a12-bc77-519cffec1c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:22.423182 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-026c4e60-0292-4a12-bc77-519cffec1c0e None None] MFA Rules not processed for user `2d62232282dc4455834857d5290eb169`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:22.426399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-026c4e60-0292-4a12-bc77-519cffec1c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:22.441745 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:22.744229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309e944-1b73-4dfb-902a-b102c570bd50 service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:22.745849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a309e944-1b73-4dfb-902a-b102c570bd50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:22.746584 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1118/2220] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:22.914807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309e944-1b73-4dfb-902a-b102c570bd50 service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:22.920435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd6533-58f8-4b01-b1bf-34e5a1d5bd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:22.921316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd6533-58f8-4b01-b1bf-34e5a1d5bd1d admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:22.921600 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd6533-58f8-4b01-b1bf-34e5a1d5bd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:22.921878 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd6533-58f8-4b01-b1bf-34e5a1d5bd1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:22.923153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd6533-58f8-4b01-b1bf-34e5a1d5bd1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:22.925819 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd6533-58f8-4b01-b1bf-34e5a1d5bd1d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:22.960514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1103/2221] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:30:22 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:22.964493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c4e60-0292-4a12-bc77-519cffec1c0e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:22.987309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c4e60-0292-4a12-bc77-519cffec1c0e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:23.082895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49436da3-5d18-4b11-a045-087cd1f316ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:23.083825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49436da3-5d18-4b11-a045-087cd1f316ab admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:23.084121 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49436da3-5d18-4b11-a045-087cd1f316ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:23.084534 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49436da3-5d18-4b11-a045-087cd1f316ab admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:23.097881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49436da3-5d18-4b11-a045-087cd1f316ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:23.099863 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49436da3-5d18-4b11-a045-087cd1f316ab admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:23.131678 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1119/2222] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:30:22 2022] POST /identity/v3/users => generated 295 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:30:23.132917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd6533-58f8-4b01-b1bf-34e5a1d5bd1d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:23.144970 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd6533-58f8-4b01-b1bf-34e5a1d5bd1d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:23.149728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92934be-b8aa-492c-af3b-c7d8e33bf4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:23.150328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92934be-b8aa-492c-af3b-c7d8e33bf4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:23.150608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92934be-b8aa-492c-af3b-c7d8e33bf4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:23.150880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92934be-b8aa-492c-af3b-c7d8e33bf4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:23.151518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92934be-b8aa-492c-af3b-c7d8e33bf4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:23.153157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92934be-b8aa-492c-af3b-c7d8e33bf4d4 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:23.164725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1104/2223] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:23.168797 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49436da3-5d18-4b11-a045-087cd1f316ab admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:23.180358 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49436da3-5d18-4b11-a045-087cd1f316ab admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:23.185731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840116d-3556-4cfa-a977-984b84c493db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:23.186404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840116d-3556-4cfa-a977-984b84c493db admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:23.186710 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840116d-3556-4cfa-a977-984b84c493db admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:23.188028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840116d-3556-4cfa-a977-984b84c493db admin admin] PATH_INFO: `/v3/projects/90b1f65e3cf44a8fae0aed69971e8810/users/97719914a8b34a7496e718d40865487b/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:23.208186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840116d-3556-4cfa-a977-984b84c493db admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b1f65e3cf44a8fae0aed69971e8810, user_id=97719914a8b34a7496e718d40865487b, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:23.211420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840116d-3556-4cfa-a977-984b84c493db admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:23.232815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1120/2224] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:23 2022] PUT /identity/v3/projects/90b1f65e3cf44a8fae0aed69971e8810/users/97719914a8b34a7496e718d40865487b/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:23.236916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92934be-b8aa-492c-af3b-c7d8e33bf4d4 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:23.247671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92934be-b8aa-492c-af3b-c7d8e33bf4d4 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:23.306863 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32024d9c-3ad0-48fd-bf92-596c836f4563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:23.307636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32024d9c-3ad0-48fd-bf92-596c836f4563 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:23.307923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32024d9c-3ad0-48fd-bf92-596c836f4563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:23.308199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32024d9c-3ad0-48fd-bf92-596c836f4563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:23.308838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32024d9c-3ad0-48fd-bf92-596c836f4563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:23.310662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32024d9c-3ad0-48fd-bf92-596c836f4563 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:23.323658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1105/2225] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:23.326222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840116d-3556-4cfa-a977-984b84c493db admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:23.338330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840116d-3556-4cfa-a977-984b84c493db admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:23.343478 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ed6d3-45bf-4060-acc6-55798dfed384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:23.344351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ed6d3-45bf-4060-acc6-55798dfed384 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:23.344632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ed6d3-45bf-4060-acc6-55798dfed384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:23.344908 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ed6d3-45bf-4060-acc6-55798dfed384 admin admin] PATH_INFO: `/v3/projects/90b1f65e3cf44a8fae0aed69971e8810/users/97719914a8b34a7496e718d40865487b/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:23.346837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ed6d3-45bf-4060-acc6-55798dfed384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b1f65e3cf44a8fae0aed69971e8810, user_id=97719914a8b34a7496e718d40865487b, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:23.349711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ed6d3-45bf-4060-acc6-55798dfed384 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:23.360593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1121/2226] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:23 2022] PUT /identity/v3/projects/90b1f65e3cf44a8fae0aed69971e8810/users/97719914a8b34a7496e718d40865487b/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:30:29.152678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39baae-dd38-44e0-8e10-59511d956cb6 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:29.154456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39baae-dd38-44e0-8e10-59511d956cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:29.155137 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39baae-dd38-44e0-8e10-59511d956cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:29.206023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-4d39baae-dd38-44e0-8e10-59511d956cb6 None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:29.207878 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4d39baae-dd38-44e0-8e10-59511d956cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:29.339341 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:29.405408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1106/2227] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:30:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:29.541056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ed6d3-45bf-4060-acc6-55798dfed384 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:29.544047 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ca4ed6d3-45bf-4060-acc6-55798dfed384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:29.700238 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:29.728794 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ed6d3-45bf-4060-acc6-55798dfed384 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:29.738454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d49a4a-8d36-441e-85cc-c645f9911221 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:29.739108 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d49a4a-8d36-441e-85cc-c645f9911221 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:29.739413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d49a4a-8d36-441e-85cc-c645f9911221 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:29.739747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d49a4a-8d36-441e-85cc-c645f9911221 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:29.755540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d49a4a-8d36-441e-85cc-c645f9911221 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:29.758173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d49a4a-8d36-441e-85cc-c645f9911221 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:29.770586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1122/2228] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:30:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:30:29.935867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22263c4-82a8-4e07-86bb-9a5f8cd8d82f None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:29.936163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22263c4-82a8-4e07-86bb-9a5f8cd8d82f None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:29.936449 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22263c4-82a8-4e07-86bb-9a5f8cd8d82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:30.044841 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c22263c4-82a8-4e07-86bb-9a5f8cd8d82f None None] MFA Rules not processed for user `97719914a8b34a7496e718d40865487b`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:30.047736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c22263c4-82a8-4e07-86bb-9a5f8cd8d82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:30.117204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d49a4a-8d36-441e-85cc-c645f9911221 service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:30.120639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-92d49a4a-8d36-441e-85cc-c645f9911221 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:30.293620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1107/2229] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:30.293620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22263c4-82a8-4e07-86bb-9a5f8cd8d82f None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:30.312908 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d49a4a-8d36-441e-85cc-c645f9911221 service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:30.340999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22263c4-82a8-4e07-86bb-9a5f8cd8d82f None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:30.350996 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4292e58-8d6f-4ef6-92e6-3e6b0e1707f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:30.353160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4292e58-8d6f-4ef6-92e6-3e6b0e1707f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:30.354592 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4292e58-8d6f-4ef6-92e6-3e6b0e1707f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:30.356692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4292e58-8d6f-4ef6-92e6-3e6b0e1707f6 admin admin] PATH_INFO: `/v3/users/b14c600226a1444fbaa0e82971f40deb` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:30.358515 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4292e58-8d6f-4ef6-92e6-3e6b0e1707f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b14c600226a1444fbaa0e82971f40deb)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:30.360058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f5f7f-c800-4c38-aa65-a218ff54a591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:30.360858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f5f7f-c800-4c38-aa65-a218ff54a591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:30.363517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f5f7f-c800-4c38-aa65-a218ff54a591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:30.363865 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f5f7f-c800-4c38-aa65-a218ff54a591 admin admin] PATH_INFO: `/v3/users/e248f668495943ba8eaa115d22349fbf` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:30.366689 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4292e58-8d6f-4ef6-92e6-3e6b0e1707f6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:30.368112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f5f7f-c800-4c38-aa65-a218ff54a591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e248f668495943ba8eaa115d22349fbf)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:30.370480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f5f7f-c800-4c38-aa65-a218ff54a591 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:30.494122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-df2f5f7f-c800-4c38-aa65-a218ff54a591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e248f668495943ba8eaa115d22349fbf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:30.528698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f4292e58-8d6f-4ef6-92e6-3e6b0e1707f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b14c600226a1444fbaa0e82971f40deb'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:30.530319 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-df2f5f7f-c800-4c38-aa65-a218ff54a591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e248f668495943ba8eaa115d22349fbf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:30.540819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f4292e58-8d6f-4ef6-92e6-3e6b0e1707f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b14c600226a1444fbaa0e82971f40deb'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:30.563285 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-df2f5f7f-c800-4c38-aa65-a218ff54a591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e248f668495943ba8eaa115d22349fbf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:30.564440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-df2f5f7f-c800-4c38-aa65-a218ff54a591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e248f668495943ba8eaa115d22349fbf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:30.583615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-df2f5f7f-c800-4c38-aa65-a218ff54a591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e248f668495943ba8eaa115d22349fbf'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:30.591536 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f4292e58-8d6f-4ef6-92e6-3e6b0e1707f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b14c600226a1444fbaa0e82971f40deb'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:30.602912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1108/2230] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:30 2022] DELETE /identity/v3/users/e248f668495943ba8eaa115d22349fbf => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:30.602912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f5f7f-c800-4c38-aa65-a218ff54a591 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:30.605333 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-df2f5f7f-c800-4c38-aa65-a218ff54a591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:30.630126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f4292e58-8d6f-4ef6-92e6-3e6b0e1707f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b14c600226a1444fbaa0e82971f40deb'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:30.634905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f4292e58-8d6f-4ef6-92e6-3e6b0e1707f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b14c600226a1444fbaa0e82971f40deb'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:30.646053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1123/2231] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:30 2022] DELETE /identity/v3/users/b14c600226a1444fbaa0e82971f40deb => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:30.650121 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5167c-7605-484e-9867-acda262dbb0f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:30.651473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5167c-7605-484e-9867-acda262dbb0f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:30.653723 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5167c-7605-484e-9867-acda262dbb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:30.721930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6db5167c-7605-484e-9867-acda262dbb0f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:30.723313 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6db5167c-7605-484e-9867-acda262dbb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:30.806102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:30.939269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f5f7f-c800-4c38-aa65-a218ff54a591 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:30.942956 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1124/2232] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:30.945485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cd719-8af2-43ac-a001-4a3d46ad9d44 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:30.955966 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cd719-8af2-43ac-a001-4a3d46ad9d44 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:30.956208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cd719-8af2-43ac-a001-4a3d46ad9d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:30.994692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:31.018499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-594cd719-8af2-43ac-a001-4a3d46ad9d44 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:31.020178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-594cd719-8af2-43ac-a001-4a3d46ad9d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:31.032519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1125/2233] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:31.034001 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cd719-8af2-43ac-a001-4a3d46ad9d44 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:31.042093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-594cd719-8af2-43ac-a001-4a3d46ad9d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:31.147668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199953e0-1d2e-4a0b-a347-18b477367a5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:31.149283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199953e0-1d2e-4a0b-a347-18b477367a5d service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:31.149746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199953e0-1d2e-4a0b-a347-18b477367a5d service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:31.150084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199953e0-1d2e-4a0b-a347-18b477367a5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:31.153540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cd719-8af2-43ac-a001-4a3d46ad9d44 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:31.156473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-199953e0-1d2e-4a0b-a347-18b477367a5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:31.161761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d192c8-2b26-404a-b592-730a300bdfe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:31.164509 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d192c8-2b26-404a-b592-730a300bdfe5 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:31.164912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d192c8-2b26-404a-b592-730a300bdfe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:31.165264 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d192c8-2b26-404a-b592-730a300bdfe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:31.188976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d192c8-2b26-404a-b592-730a300bdfe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:31.190609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d192c8-2b26-404a-b592-730a300bdfe5 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:31.214329 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1126/2234] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:31.230398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d192c8-2b26-404a-b592-730a300bdfe5 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:31.259676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d192c8-2b26-404a-b592-730a300bdfe5 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:31.325031 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cea024-708f-44f2-8afd-fbf0bc04f3a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:31.325646 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cea024-708f-44f2-8afd-fbf0bc04f3a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:31.325925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cea024-708f-44f2-8afd-fbf0bc04f3a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:31.332109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cea024-708f-44f2-8afd-fbf0bc04f3a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:31.342604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199953e0-1d2e-4a0b-a347-18b477367a5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:31.373834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199953e0-1d2e-4a0b-a347-18b477367a5d service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:31.377453 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cea024-708f-44f2-8afd-fbf0bc04f3a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:31.385784 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cea024-708f-44f2-8afd-fbf0bc04f3a0 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:31.413470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1109/2235] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 812 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:31.459308 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1127/2236] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:32.932645 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199953e0-1d2e-4a0b-a347-18b477367a5d service nova] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:32.940991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-199953e0-1d2e-4a0b-a347-18b477367a5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:33.017110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199953e0-1d2e-4a0b-a347-18b477367a5d service nova] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:33.043144 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:33.044565 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:33.044800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:33.045081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] PATH_INFO: `/v3/projects/9133ab50b25247bca270278c37632237` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:33.049222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9133ab50b25247bca270278c37632237)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:33.051223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:33.188803 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cea024-708f-44f2-8afd-fbf0bc04f3a0 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:33.191196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d9cea024-708f-44f2-8afd-fbf0bc04f3a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:33.219265 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cea024-708f-44f2-8afd-fbf0bc04f3a0 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:33.272694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9133ab50b25247bca270278c37632237', 'request_id': 'req-f062f31d-bf53-4948-baf6-f299e161238e'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.289063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9133ab50b25247bca270278c37632237', 'request_id': 'req-f062f31d-bf53-4948-baf6-f299e161238e'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.306104 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9133ab50b25247bca270278c37632237', 'request_id': 'req-f062f31d-bf53-4948-baf6-f299e161238e'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.311672 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:33.331212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] The token cache is being invalidate because project 9133ab50b25247bca270278c37632237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:33.331520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.336591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.344924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1110/2237] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:32 2022] DELETE /identity/v3/projects/9133ab50b25247bca270278c37632237 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:33.352112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:33.356349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:33.375123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:33.376231 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:33.376742 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:33.377273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] PATH_INFO: `/v3/projects/27a70cfc525344cc97780ba1238e350a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:33.378621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27a70cfc525344cc97780ba1238e350a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:33.381422 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:33.399525 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062f31d-bf53-4948-baf6-f299e161238e admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:33.418626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9811d4-8c3a-43b5-ae0b-30fab8e79873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:33.419327 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9811d4-8c3a-43b5-ae0b-30fab8e79873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:33.419662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9811d4-8c3a-43b5-ae0b-30fab8e79873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:33.419945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9811d4-8c3a-43b5-ae0b-30fab8e79873 admin admin] PATH_INFO: `/v3/users/e3536ec4edfe4e1aa3dc77fdf7bf110f` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:33.422821 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9811d4-8c3a-43b5-ae0b-30fab8e79873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3536ec4edfe4e1aa3dc77fdf7bf110f)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:33.429119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9811d4-8c3a-43b5-ae0b-30fab8e79873 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:33.469950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27a70cfc525344cc97780ba1238e350a', 'request_id': 'req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.488259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27a70cfc525344cc97780ba1238e350a', 'request_id': 'req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.512659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27a70cfc525344cc97780ba1238e350a', 'request_id': 'req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.541483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] The token cache is being invalidate because project 27a70cfc525344cc97780ba1238e350a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:33.542087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.542993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.578230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1128/2238] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:33 2022] DELETE /identity/v3/projects/27a70cfc525344cc97780ba1238e350a => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:33.596335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6c9811d4-8c3a-43b5-ae0b-30fab8e79873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3536ec4edfe4e1aa3dc77fdf7bf110f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.657304 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6c9811d4-8c3a-43b5-ae0b-30fab8e79873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3536ec4edfe4e1aa3dc77fdf7bf110f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.676234 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6c9811d4-8c3a-43b5-ae0b-30fab8e79873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3536ec4edfe4e1aa3dc77fdf7bf110f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.678639 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6c9811d4-8c3a-43b5-ae0b-30fab8e79873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3536ec4edfe4e1aa3dc77fdf7bf110f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.709508 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6c9811d4-8c3a-43b5-ae0b-30fab8e79873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3536ec4edfe4e1aa3dc77fdf7bf110f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:33.726988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1111/2239] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:33 2022] DELETE /identity/v3/users/e3536ec4edfe4e1aa3dc77fdf7bf110f => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:34.999996 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:35.001531 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:35.043525 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:35.188975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2edbc-85cf-4e2c-aecf-8fadaaacf448 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:35.396107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:35.396949 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:35.397337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:35.397717 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb admin admin] PATH_INFO: `/v3/projects/02834a352fd8489b85f5016011529f3f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:35.402409 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=02834a352fd8489b85f5016011529f3f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:35.410968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:35.552674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02834a352fd8489b85f5016011529f3f', 'request_id': 'req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:35.579181 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02834a352fd8489b85f5016011529f3f', 'request_id': 'req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:35.614433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02834a352fd8489b85f5016011529f3f', 'request_id': 'req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:35.654941 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb admin admin] The token cache is being invalidate because project 02834a352fd8489b85f5016011529f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:35.655226 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:35.662529 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6224ca7d-1aa0-46e5-845b-73c1bf6727eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:35.676351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1129/2240] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:34 2022] DELETE /identity/v3/projects/02834a352fd8489b85f5016011529f3f => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:35.682081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0940d4-f018-4a36-a53b-cd8038708f43 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:35.682650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0940d4-f018-4a36-a53b-cd8038708f43 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:35.682893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0940d4-f018-4a36-a53b-cd8038708f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:35.747926 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-cb0940d4-f018-4a36-a53b-cd8038708f43 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:35.754527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-cb0940d4-f018-4a36-a53b-cd8038708f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:35.847838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1112/2241] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:35.857129 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ed8a9-6af2-44b2-bd5f-8678d8c6b7f5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:35.857422 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ed8a9-6af2-44b2-bd5f-8678d8c6b7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:35.858337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ed8a9-6af2-44b2-bd5f-8678d8c6b7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:35.964675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-686ed8a9-6af2-44b2-bd5f-8678d8c6b7f5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:35.966547 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-686ed8a9-6af2-44b2-bd5f-8678d8c6b7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:36.035293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1130/2242] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:30:36.040142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0940d4-f018-4a36-a53b-cd8038708f43 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:36.053454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0940d4-f018-4a36-a53b-cd8038708f43 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:36.073117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a0f11-9a47-4010-80ea-858d280fc968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:36.073790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a0f11-9a47-4010-80ea-858d280fc968 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:36.084398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a0f11-9a47-4010-80ea-858d280fc968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:36.084760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a0f11-9a47-4010-80ea-858d280fc968 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:36.086283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a0f11-9a47-4010-80ea-858d280fc968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:30:36.086599 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a0f11-9a47-4010-80ea-858d280fc968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:36.092228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a0f11-9a47-4010-80ea-858d280fc968 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:36.123205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1113/2243] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:30:36.127645 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ed8a9-6af2-44b2-bd5f-8678d8c6b7f5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:36.146921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ed8a9-6af2-44b2-bd5f-8678d8c6b7f5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:36.163561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607520b6-0a78-4fc8-852f-ac8ff2bb6e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:36.164505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607520b6-0a78-4fc8-852f-ac8ff2bb6e55 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:36.164956 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607520b6-0a78-4fc8-852f-ac8ff2bb6e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:36.165390 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607520b6-0a78-4fc8-852f-ac8ff2bb6e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:36.166199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607520b6-0a78-4fc8-852f-ac8ff2bb6e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:36.176732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607520b6-0a78-4fc8-852f-ac8ff2bb6e55 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:36.286967 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1131/2244] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:30:36 2022] POST /identity/v3/projects => generated 381 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:36.291456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a0f11-9a47-4010-80ea-858d280fc968 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:36.319790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a0f11-9a47-4010-80ea-858d280fc968 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:36.490679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:36.508040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045717d-c06d-4ab3-806e-21bed032e1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:36.509768 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045717d-c06d-4ab3-806e-21bed032e1fe admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:36.512572 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045717d-c06d-4ab3-806e-21bed032e1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:36.513463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045717d-c06d-4ab3-806e-21bed032e1fe admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:36.514244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045717d-c06d-4ab3-806e-21bed032e1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:36.517825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045717d-c06d-4ab3-806e-21bed032e1fe admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:36.569879 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1114/2245] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:30:36 2022] POST /identity/v3/users => generated 291 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:36.573993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607520b6-0a78-4fc8-852f-ac8ff2bb6e55 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:36.592123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607520b6-0a78-4fc8-852f-ac8ff2bb6e55 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:36.610303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01577593-8291-4c58-b929-124549d04ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:36.612109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01577593-8291-4c58-b929-124549d04ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:36.612639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01577593-8291-4c58-b929-124549d04ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:36.613807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01577593-8291-4c58-b929-124549d04ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:36.614766 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01577593-8291-4c58-b929-124549d04ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:36.620421 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01577593-8291-4c58-b929-124549d04ed8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:36.647957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1132/2246] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:36.652415 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045717d-c06d-4ab3-806e-21bed032e1fe admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:36.678733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045717d-c06d-4ab3-806e-21bed032e1fe admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:36.691732 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df445737-54b9-43a9-b38a-7382c0e31818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:36.693492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df445737-54b9-43a9-b38a-7382c0e31818 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:36.695519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df445737-54b9-43a9-b38a-7382c0e31818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:36.697241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df445737-54b9-43a9-b38a-7382c0e31818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:36.712499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df445737-54b9-43a9-b38a-7382c0e31818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:36.717142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df445737-54b9-43a9-b38a-7382c0e31818 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:36.735255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1115/2247] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:36.740778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01577593-8291-4c58-b929-124549d04ed8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:36.759420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01577593-8291-4c58-b929-124549d04ed8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:36.769138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a9478-64ea-49d6-8523-56d6d2c45330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:36.769782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a9478-64ea-49d6-8523-56d6d2c45330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:36.770233 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a9478-64ea-49d6-8523-56d6d2c45330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:36.770668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a9478-64ea-49d6-8523-56d6d2c45330 admin admin] PATH_INFO: `/v3/projects/b7202f46f9c043bd8ec54bd72ec00b66/users/55acd4a559a7431bb7af5c28ee8d1d05/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:36.806893 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a9478-64ea-49d6-8523-56d6d2c45330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7202f46f9c043bd8ec54bd72ec00b66, user_id=55acd4a559a7431bb7af5c28ee8d1d05, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:36.810156 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a9478-64ea-49d6-8523-56d6d2c45330 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:36.812577 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:36.830928 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1133/2248] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:36 2022] PUT /identity/v3/projects/b7202f46f9c043bd8ec54bd72ec00b66/users/55acd4a559a7431bb7af5c28ee8d1d05/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:36.842470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea5e9d-8ab5-449f-bac8-9ba080365236 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:36.843021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea5e9d-8ab5-449f-bac8-9ba080365236 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:36.843472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea5e9d-8ab5-449f-bac8-9ba080365236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:36.901101 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-10ea5e9d-8ab5-449f-bac8-9ba080365236 None None] MFA Rules not processed for user `55acd4a559a7431bb7af5c28ee8d1d05`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:36.902579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-10ea5e9d-8ab5-449f-bac8-9ba080365236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:37.178650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1116/2249] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:37.180023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a9478-64ea-49d6-8523-56d6d2c45330 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:37.404646 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a9478-64ea-49d6-8523-56d6d2c45330 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:37.422420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd319954-8a2e-4bc6-8297-5fb50b522db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:37.423259 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd319954-8a2e-4bc6-8297-5fb50b522db0 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:37.425266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd319954-8a2e-4bc6-8297-5fb50b522db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:37.425725 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd319954-8a2e-4bc6-8297-5fb50b522db0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:37.426537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd319954-8a2e-4bc6-8297-5fb50b522db0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:37.429651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd319954-8a2e-4bc6-8297-5fb50b522db0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:37.518109 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea5e9d-8ab5-449f-bac8-9ba080365236 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:37.531796 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1134/2250] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:30:37 2022] POST /identity/v3/projects => generated 379 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:37.538209 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea5e9d-8ab5-449f-bac8-9ba080365236 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:37.705869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24288b0-8af7-4ff9-8669-73d5b1385939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:37.707351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24288b0-8af7-4ff9-8669-73d5b1385939 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:37.708068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24288b0-8af7-4ff9-8669-73d5b1385939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:37.708494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24288b0-8af7-4ff9-8669-73d5b1385939 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:37.709290 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24288b0-8af7-4ff9-8669-73d5b1385939 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:37.712737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24288b0-8af7-4ff9-8669-73d5b1385939 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:37.767991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd319954-8a2e-4bc6-8297-5fb50b522db0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:37.774010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1117/2251] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:30:37 2022] POST /identity/v3/users => generated 296 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:37.784201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd319954-8a2e-4bc6-8297-5fb50b522db0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:37.799470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa5219-7a62-41ed-8e94-4bef2c2378ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:37.800342 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa5219-7a62-41ed-8e94-4bef2c2378ed admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:37.800789 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa5219-7a62-41ed-8e94-4bef2c2378ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:37.801215 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa5219-7a62-41ed-8e94-4bef2c2378ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:37.802083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa5219-7a62-41ed-8e94-4bef2c2378ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:37.803886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa5219-7a62-41ed-8e94-4bef2c2378ed admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:37.834124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24288b0-8af7-4ff9-8669-73d5b1385939 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:37.844731 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24288b0-8af7-4ff9-8669-73d5b1385939 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:37.848706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1135/2252] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:30:37.865381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e599d-da86-4d23-afe5-c76961a45859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:37.866082 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e599d-da86-4d23-afe5-c76961a45859 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:37.866588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e599d-da86-4d23-afe5-c76961a45859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:37.867037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e599d-da86-4d23-afe5-c76961a45859 admin admin] PATH_INFO: `/v3/projects/57f54fbbcd724f40b2306afb1b6df738/users/d6f2646471a245a2b1f31ac19aeb6eab/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:37.907172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e599d-da86-4d23-afe5-c76961a45859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f54fbbcd724f40b2306afb1b6df738, user_id=d6f2646471a245a2b1f31ac19aeb6eab, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:37.910713 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e599d-da86-4d23-afe5-c76961a45859 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:37.945649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa5219-7a62-41ed-8e94-4bef2c2378ed admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:37.946384 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1118/2253] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:37 2022] PUT /identity/v3/projects/57f54fbbcd724f40b2306afb1b6df738/users/d6f2646471a245a2b1f31ac19aeb6eab/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:37.961179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa5219-7a62-41ed-8e94-4bef2c2378ed admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:38.102114 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabcc5f-12e7-4f27-a330-9bab72f5812e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:38.103022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabcc5f-12e7-4f27-a330-9bab72f5812e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:38.103506 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabcc5f-12e7-4f27-a330-9bab72f5812e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:38.104000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabcc5f-12e7-4f27-a330-9bab72f5812e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:38.104813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabcc5f-12e7-4f27-a330-9bab72f5812e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:38.107261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabcc5f-12e7-4f27-a330-9bab72f5812e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:38.141334 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e599d-da86-4d23-afe5-c76961a45859 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:38.141788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1136/2254] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:38.159540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e599d-da86-4d23-afe5-c76961a45859 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:38.173628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34549af1-529b-4ddb-b891-2027bd2a57c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:38.174452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34549af1-529b-4ddb-b891-2027bd2a57c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:38.174751 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34549af1-529b-4ddb-b891-2027bd2a57c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:38.175137 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34549af1-529b-4ddb-b891-2027bd2a57c8 admin admin] PATH_INFO: `/v3/projects/57f54fbbcd724f40b2306afb1b6df738/users/d6f2646471a245a2b1f31ac19aeb6eab/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:38.181161 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34549af1-529b-4ddb-b891-2027bd2a57c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f54fbbcd724f40b2306afb1b6df738, user_id=d6f2646471a245a2b1f31ac19aeb6eab, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:38.193164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34549af1-529b-4ddb-b891-2027bd2a57c8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:38.248602 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c179a4e-b140-4925-b2a1-81b20007de7d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:38.249545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c179a4e-b140-4925-b2a1-81b20007de7d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:38.249828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c179a4e-b140-4925-b2a1-81b20007de7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:38.264829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1119/2255] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:38 2022] PUT /identity/v3/projects/57f54fbbcd724f40b2306afb1b6df738/users/d6f2646471a245a2b1f31ac19aeb6eab/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:38.334976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7c179a4e-b140-4925-b2a1-81b20007de7d None None] MFA Rules not processed for user `d6f2646471a245a2b1f31ac19aeb6eab`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:38.347591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7c179a4e-b140-4925-b2a1-81b20007de7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:38.530246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:38.588367 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1137/2256] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:38.601633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34549af1-529b-4ddb-b891-2027bd2a57c8 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:38.605055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-34549af1-529b-4ddb-b891-2027bd2a57c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:38.676619 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:38.842662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34549af1-529b-4ddb-b891-2027bd2a57c8 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:38.895348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8622602-dca8-4cd3-8aa6-f429cacbf86f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:38.896490 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8622602-dca8-4cd3-8aa6-f429cacbf86f service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:38.903704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8622602-dca8-4cd3-8aa6-f429cacbf86f service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:38.915866 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8622602-dca8-4cd3-8aa6-f429cacbf86f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:38.958434 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8622602-dca8-4cd3-8aa6-f429cacbf86f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:38.961374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8622602-dca8-4cd3-8aa6-f429cacbf86f service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:39.003644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1120/2257] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:40.120872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c179a4e-b140-4925-b2a1-81b20007de7d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:40.257406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c179a4e-b140-4925-b2a1-81b20007de7d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:40.266994 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac24d8a-34ee-40e7-b364-d1eff43de2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:40.274324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac24d8a-34ee-40e7-b364-d1eff43de2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:40.274324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac24d8a-34ee-40e7-b364-d1eff43de2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:40.274324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac24d8a-34ee-40e7-b364-d1eff43de2f0 admin admin] PATH_INFO: `/v3/users/55acd4a559a7431bb7af5c28ee8d1d05` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:40.274324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac24d8a-34ee-40e7-b364-d1eff43de2f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55acd4a559a7431bb7af5c28ee8d1d05)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:40.276351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac24d8a-34ee-40e7-b364-d1eff43de2f0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:40.436699 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fac24d8a-34ee-40e7-b364-d1eff43de2f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55acd4a559a7431bb7af5c28ee8d1d05'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:40.468002 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fac24d8a-34ee-40e7-b364-d1eff43de2f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55acd4a559a7431bb7af5c28ee8d1d05'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:40.528362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fac24d8a-34ee-40e7-b364-d1eff43de2f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55acd4a559a7431bb7af5c28ee8d1d05'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:40.541049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fac24d8a-34ee-40e7-b364-d1eff43de2f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55acd4a559a7431bb7af5c28ee8d1d05'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:40.568549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fac24d8a-34ee-40e7-b364-d1eff43de2f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55acd4a559a7431bb7af5c28ee8d1d05'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:40.613675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1138/2258] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:40 2022] DELETE /identity/v3/users/55acd4a559a7431bb7af5c28ee8d1d05 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:40.618385 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f40d9-b09d-4cdf-ac2f-0c8cdfe0cb21 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:40.619225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f40d9-b09d-4cdf-ac2f-0c8cdfe0cb21 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:40.619713 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f40d9-b09d-4cdf-ac2f-0c8cdfe0cb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:40.670194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac24d8a-34ee-40e7-b364-d1eff43de2f0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:40.676469 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fac24d8a-34ee-40e7-b364-d1eff43de2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:40.698448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b67f40d9-b09d-4cdf-ac2f-0c8cdfe0cb21 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:40.704903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b67f40d9-b09d-4cdf-ac2f-0c8cdfe0cb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:40.807818 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac24d8a-34ee-40e7-b364-d1eff43de2f0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:40.832527 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bb4cc-d88e-4e99-9e6f-c1e3258c15a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:40.833437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bb4cc-d88e-4e99-9e6f-c1e3258c15a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:40.833717 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bb4cc-d88e-4e99-9e6f-c1e3258c15a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:40.833999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bb4cc-d88e-4e99-9e6f-c1e3258c15a5 admin admin] PATH_INFO: `/v3/users/cd1582f15be0450ca1e461251a2b515c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:40.835323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bb4cc-d88e-4e99-9e6f-c1e3258c15a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd1582f15be0450ca1e461251a2b515c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:40.838673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bb4cc-d88e-4e99-9e6f-c1e3258c15a5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:40.881495 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:40.907095 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1121/2259] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:40.918374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f40d9-b09d-4cdf-ac2f-0c8cdfe0cb21 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:40.920542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b67f40d9-b09d-4cdf-ac2f-0c8cdfe0cb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:40.989350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e63bb4cc-d88e-4e99-9e6f-c1e3258c15a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd1582f15be0450ca1e461251a2b515c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:41.005721 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e63bb4cc-d88e-4e99-9e6f-c1e3258c15a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd1582f15be0450ca1e461251a2b515c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:41.023133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e63bb4cc-d88e-4e99-9e6f-c1e3258c15a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd1582f15be0450ca1e461251a2b515c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:41.039796 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e63bb4cc-d88e-4e99-9e6f-c1e3258c15a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd1582f15be0450ca1e461251a2b515c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:41.041803 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e63bb4cc-d88e-4e99-9e6f-c1e3258c15a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd1582f15be0450ca1e461251a2b515c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:41.062890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1139/2260] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:40 2022] DELETE /identity/v3/users/cd1582f15be0450ca1e461251a2b515c => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:41.067150 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c6141-5316-43e6-a98e-85025a1e1534 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:41.068003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c6141-5316-43e6-a98e-85025a1e1534 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:41.068440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c6141-5316-43e6-a98e-85025a1e1534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:41.074553 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f40d9-b09d-4cdf-ac2f-0c8cdfe0cb21 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:41.127600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-246c6141-5316-43e6-a98e-85025a1e1534 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:41.133047 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-246c6141-5316-43e6-a98e-85025a1e1534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:41.282168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d784fd4-4960-4174-a985-6dd046c08fb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:41.283077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784fd4-4960-4174-a985-6dd046c08fb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:41.283560 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784fd4-4960-4174-a985-6dd046c08fb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:41.284049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784fd4-4960-4174-a985-6dd046c08fb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:41.291022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5d784fd4-4960-4174-a985-6dd046c08fb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:41.366309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1140/2261] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:41.376170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c6141-5316-43e6-a98e-85025a1e1534 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:41.378803 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-246c6141-5316-43e6-a98e-85025a1e1534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:41.408315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c6141-5316-43e6-a98e-85025a1e1534 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:41.409530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:41.422799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb30a16-c011-4c91-9d19-5509cf569139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:41.423471 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb30a16-c011-4c91-9d19-5509cf569139 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:41.423881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb30a16-c011-4c91-9d19-5509cf569139 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:41.424258 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb30a16-c011-4c91-9d19-5509cf569139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:41.436380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb30a16-c011-4c91-9d19-5509cf569139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:41.438234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb30a16-c011-4c91-9d19-5509cf569139 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:41.487798 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1141/2262] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:41.546126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d784fd4-4960-4174-a985-6dd046c08fb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:41.548608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d784fd4-4960-4174-a985-6dd046c08fb8 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:41.572664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1122/2263] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:30:42.170516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb30a16-c011-4c91-9d19-5509cf569139 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:42.172949 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7eb30a16-c011-4c91-9d19-5509cf569139 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:42.200399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb30a16-c011-4c91-9d19-5509cf569139 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:42.209988 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:42.210922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:42.213965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:42.214160 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] PATH_INFO: `/v3/projects/d6e9ad6b0fd043be84d8f82f276e913f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:42.216269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6e9ad6b0fd043be84d8f82f276e913f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:42.218462 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:42.376793 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6e9ad6b0fd043be84d8f82f276e913f', 'request_id': 'req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:42.396399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6e9ad6b0fd043be84d8f82f276e913f', 'request_id': 'req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:42.412487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6e9ad6b0fd043be84d8f82f276e913f', 'request_id': 'req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:42.431742 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:42.448151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] The token cache is being invalidate because project d6e9ad6b0fd043be84d8f82f276e913f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:42.448697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:42.467817 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:42.470934 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d784fd4-4960-4174-a985-6dd046c08fb8 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:42.471451 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5d784fd4-4960-4174-a985-6dd046c08fb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:42.482217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1142/2264] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:42 2022] DELETE /identity/v3/projects/d6e9ad6b0fd043be84d8f82f276e913f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:42.516795 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d784fd4-4960-4174-a985-6dd046c08fb8 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:42.777808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:42.780885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:42.781420 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:42.783069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] PATH_INFO: `/v3/projects/b7202f46f9c043bd8ec54bd72ec00b66` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:42.787793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7202f46f9c043bd8ec54bd72ec00b66)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:42.790246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:42.954468 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7202f46f9c043bd8ec54bd72ec00b66', 'request_id': 'req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:42.973928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7202f46f9c043bd8ec54bd72ec00b66', 'request_id': 'req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:42.993105 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7202f46f9c043bd8ec54bd72ec00b66', 'request_id': 'req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:43.018275 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] The token cache is being invalidate because project b7202f46f9c043bd8ec54bd72ec00b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:43.021868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:43.026091 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:43.030852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1123/2265] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:42 2022] DELETE /identity/v3/projects/b7202f46f9c043bd8ec54bd72ec00b66 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:43.034843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:43.036338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:43.088276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9ad4c-93c1-475a-ad6e-bd1b8a7edfaf admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:43.310554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf0aec-333d-4e14-85f8-c1d1924c3ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:43.312284 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf0aec-333d-4e14-85f8-c1d1924c3ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:43.315163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf0aec-333d-4e14-85f8-c1d1924c3ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:43.315163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf0aec-333d-4e14-85f8-c1d1924c3ace admin admin] PATH_INFO: `/v3/users/d6f2646471a245a2b1f31ac19aeb6eab` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:43.333221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf0aec-333d-4e14-85f8-c1d1924c3ace admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6f2646471a245a2b1f31ac19aeb6eab)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:43.338018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf0aec-333d-4e14-85f8-c1d1924c3ace admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:43.466131 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ddcf0aec-333d-4e14-85f8-c1d1924c3ace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6f2646471a245a2b1f31ac19aeb6eab'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:43.482084 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ddcf0aec-333d-4e14-85f8-c1d1924c3ace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6f2646471a245a2b1f31ac19aeb6eab'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:43.525345 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ddcf0aec-333d-4e14-85f8-c1d1924c3ace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6f2646471a245a2b1f31ac19aeb6eab'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:43.545901 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ddcf0aec-333d-4e14-85f8-c1d1924c3ace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6f2646471a245a2b1f31ac19aeb6eab'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:43.546952 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ddcf0aec-333d-4e14-85f8-c1d1924c3ace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6f2646471a245a2b1f31ac19aeb6eab'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:43.571769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1143/2266] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:43 2022] DELETE /identity/v3/users/d6f2646471a245a2b1f31ac19aeb6eab => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:44.611005 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:44.613564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:44.675062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4fa5d-dfc9-4fec-b73b-4ea2b1e6d470 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:45.087394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:45.125879 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e661c-9df2-4cc9-a07e-648fb5815c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:45.126433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e661c-9df2-4cc9-a07e-648fb5815c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:45.126718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e661c-9df2-4cc9-a07e-648fb5815c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:45.127735 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e661c-9df2-4cc9-a07e-648fb5815c18 admin admin] PATH_INFO: `/v3/projects/57f54fbbcd724f40b2306afb1b6df738` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:45.147599 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e661c-9df2-4cc9-a07e-648fb5815c18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f54fbbcd724f40b2306afb1b6df738)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:45.171685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e661c-9df2-4cc9-a07e-648fb5815c18 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:45.367679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-bf1e661c-9df2-4cc9-a07e-648fb5815c18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f54fbbcd724f40b2306afb1b6df738', 'request_id': 'req-bf1e661c-9df2-4cc9-a07e-648fb5815c18'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:45.388736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-bf1e661c-9df2-4cc9-a07e-648fb5815c18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f54fbbcd724f40b2306afb1b6df738', 'request_id': 'req-bf1e661c-9df2-4cc9-a07e-648fb5815c18'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:45.401506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-bf1e661c-9df2-4cc9-a07e-648fb5815c18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f54fbbcd724f40b2306afb1b6df738', 'request_id': 'req-bf1e661c-9df2-4cc9-a07e-648fb5815c18'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:45.425329 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-bf1e661c-9df2-4cc9-a07e-648fb5815c18 admin admin] The token cache is being invalidate because project 57f54fbbcd724f40b2306afb1b6df738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:45.425635 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-bf1e661c-9df2-4cc9-a07e-648fb5815c18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:45.426561 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-bf1e661c-9df2-4cc9-a07e-648fb5815c18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:45.429127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1124/2267] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:44 2022] DELETE /identity/v3/projects/57f54fbbcd724f40b2306afb1b6df738 => generated 0 bytes in 819 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:45.435382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f29e8-4625-4398-ba45-b439e0fd1ad6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:45.435938 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f29e8-4625-4398-ba45-b439e0fd1ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:45.436445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f29e8-4625-4398-ba45-b439e0fd1ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:45.500351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7a5f29e8-4625-4398-ba45-b439e0fd1ad6 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:45.502147 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7a5f29e8-4625-4398-ba45-b439e0fd1ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:45.745051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1144/2268] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:45.755523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3f962-97cc-4efe-be1a-b2ddca059b60 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:45.756249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3f962-97cc-4efe-be1a-b2ddca059b60 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:45.759172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3f962-97cc-4efe-be1a-b2ddca059b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:45.818151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-fac3f962-97cc-4efe-be1a-b2ddca059b60 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:45.820449 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-fac3f962-97cc-4efe-be1a-b2ddca059b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:45.856854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1125/2269] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:45.867255 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f29e8-4625-4398-ba45-b439e0fd1ad6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:45.884665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f29e8-4625-4398-ba45-b439e0fd1ad6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:45.894718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220de17-16f7-4b9c-ae2f-017397056380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:45.898147 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220de17-16f7-4b9c-ae2f-017397056380 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:45.898553 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220de17-16f7-4b9c-ae2f-017397056380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:45.898978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220de17-16f7-4b9c-ae2f-017397056380 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:45.900046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220de17-16f7-4b9c-ae2f-017397056380 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:30:45.901208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220de17-16f7-4b9c-ae2f-017397056380 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:45.905675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220de17-16f7-4b9c-ae2f-017397056380 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:45.924888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1145/2270] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:30:45.929067 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3f962-97cc-4efe-be1a-b2ddca059b60 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:45.950885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3f962-97cc-4efe-be1a-b2ddca059b60 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:45.972986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4187153-edc6-4210-9862-085635bfb43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:45.975386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4187153-edc6-4210-9862-085635bfb43f admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:45.975849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4187153-edc6-4210-9862-085635bfb43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:45.976398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4187153-edc6-4210-9862-085635bfb43f admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:45.977630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4187153-edc6-4210-9862-085635bfb43f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:45.979345 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4187153-edc6-4210-9862-085635bfb43f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:46.062812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1126/2271] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:30:45 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:30:46.066757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220de17-16f7-4b9c-ae2f-017397056380 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:46.076001 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:46.081422 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220de17-16f7-4b9c-ae2f-017397056380 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:46.240570 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efc176-7733-431c-8c29-840c579a8204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:46.240982 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efc176-7733-431c-8c29-840c579a8204 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:46.241373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efc176-7733-431c-8c29-840c579a8204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:46.241759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efc176-7733-431c-8c29-840c579a8204 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:46.244225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99efc176-7733-431c-8c29-840c579a8204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:46.249651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99efc176-7733-431c-8c29-840c579a8204 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:46.299149 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1146/2272] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:30:46 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:46.302926 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4187153-edc6-4210-9862-085635bfb43f admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:46.327211 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4187153-edc6-4210-9862-085635bfb43f admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:46.332352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492df33-1433-4f10-81f4-02653d9f9573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:46.332963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492df33-1433-4f10-81f4-02653d9f9573 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:46.333243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492df33-1433-4f10-81f4-02653d9f9573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:46.333517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492df33-1433-4f10-81f4-02653d9f9573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:46.334166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492df33-1433-4f10-81f4-02653d9f9573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:46.336447 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492df33-1433-4f10-81f4-02653d9f9573 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:46.366919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1127/2273] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:46.368627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efc176-7733-431c-8c29-840c579a8204 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:46.382664 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efc176-7733-431c-8c29-840c579a8204 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:46.401588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf74b7e-f2f2-4861-859e-2332c4a34761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:46.402269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf74b7e-f2f2-4861-859e-2332c4a34761 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:46.402554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf74b7e-f2f2-4861-859e-2332c4a34761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:46.402834 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf74b7e-f2f2-4861-859e-2332c4a34761 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:46.403499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf74b7e-f2f2-4861-859e-2332c4a34761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:46.414491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf74b7e-f2f2-4861-859e-2332c4a34761 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:46.458938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492df33-1433-4f10-81f4-02653d9f9573 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:46.474595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492df33-1433-4f10-81f4-02653d9f9573 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:46.479347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1147/2274] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:46.497308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb9438-f853-4635-9607-4b0bff8cb5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:46.497996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb9438-f853-4635-9607-4b0bff8cb5f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:46.499669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb9438-f853-4635-9607-4b0bff8cb5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:46.501055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb9438-f853-4635-9607-4b0bff8cb5f1 admin admin] PATH_INFO: `/v3/projects/c947c5f05ce146acb92d80c92ebf1705/users/eb1ef4a8d3d74cb29b07fb66c5065011/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:46.554991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb9438-f853-4635-9607-4b0bff8cb5f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c947c5f05ce146acb92d80c92ebf1705, user_id=eb1ef4a8d3d74cb29b07fb66c5065011, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:46.558415 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb9438-f853-4635-9607-4b0bff8cb5f1 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:46.587026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1128/2275] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:46 2022] PUT /identity/v3/projects/c947c5f05ce146acb92d80c92ebf1705/users/eb1ef4a8d3d74cb29b07fb66c5065011/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:30:46.596953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f454e59-2e03-4714-b4bb-21238e83e26f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:46.597256 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f454e59-2e03-4714-b4bb-21238e83e26f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:46.597583 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f454e59-2e03-4714-b4bb-21238e83e26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:46.669921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8f454e59-2e03-4714-b4bb-21238e83e26f None None] MFA Rules not processed for user `eb1ef4a8d3d74cb29b07fb66c5065011`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:46.673619 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8f454e59-2e03-4714-b4bb-21238e83e26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:46.921709 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1148/2276] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:46.922934 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb9438-f853-4635-9607-4b0bff8cb5f1 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:46.978845 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a007305-b7ff-4af3-a181-d7650a6072ca None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:46.979589 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a007305-b7ff-4af3-a181-d7650a6072ca None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:46.980155 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a007305-b7ff-4af3-a181-d7650a6072ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:47.055693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6a007305-b7ff-4af3-a181-d7650a6072ca None None] MFA Rules not processed for user `0248519bf5b54ce3ba3b40f2c9a4c628`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:47.061966 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6a007305-b7ff-4af3-a181-d7650a6072ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:47.108702 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb9438-f853-4635-9607-4b0bff8cb5f1 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:47.119897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d54b8-22da-4fb2-9f71-3f62b8e175bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:47.121012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d54b8-22da-4fb2-9f71-3f62b8e175bb admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:47.121540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d54b8-22da-4fb2-9f71-3f62b8e175bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:47.122092 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d54b8-22da-4fb2-9f71-3f62b8e175bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:47.124711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d54b8-22da-4fb2-9f71-3f62b8e175bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:47.129140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d54b8-22da-4fb2-9f71-3f62b8e175bb admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:47.192838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1129/2277] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:30:46 2022] POST /identity/v3/projects => generated 377 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:47.198203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d54b8-22da-4fb2-9f71-3f62b8e175bb admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:47.231913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1149/2278] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:47.241903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:47.357011 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d54b8-22da-4fb2-9f71-3f62b8e175bb admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:47.374762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d02adb-1b3e-4a5e-832a-f1da0885a776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:47.376076 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d02adb-1b3e-4a5e-832a-f1da0885a776 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:47.376076 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d02adb-1b3e-4a5e-832a-f1da0885a776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:47.376076 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d02adb-1b3e-4a5e-832a-f1da0885a776 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:47.376674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d02adb-1b3e-4a5e-832a-f1da0885a776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:47.378655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d02adb-1b3e-4a5e-832a-f1da0885a776 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:47.429893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1130/2279] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:30:47 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:47.430820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a007305-b7ff-4af3-a181-d7650a6072ca None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:47.457561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a007305-b7ff-4af3-a181-d7650a6072ca None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:47.478178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf3f03-a94d-49f9-9aa8-dd8eaad35ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:47.480480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf3f03-a94d-49f9-9aa8-dd8eaad35ccc admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:47.481032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf3f03-a94d-49f9-9aa8-dd8eaad35ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:47.485186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf3f03-a94d-49f9-9aa8-dd8eaad35ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:47.485365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf3f03-a94d-49f9-9aa8-dd8eaad35ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:47.492123 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf3f03-a94d-49f9-9aa8-dd8eaad35ccc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:47.528335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1150/2280] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:47.534255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d02adb-1b3e-4a5e-832a-f1da0885a776 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:47.550604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d02adb-1b3e-4a5e-832a-f1da0885a776 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:47.563123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e5a0c-8032-44c9-8743-a6fdc74c2fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:47.564696 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e5a0c-8032-44c9-8743-a6fdc74c2fed admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:47.565086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e5a0c-8032-44c9-8743-a6fdc74c2fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:47.566488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e5a0c-8032-44c9-8743-a6fdc74c2fed admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:47.569624 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e5a0c-8032-44c9-8743-a6fdc74c2fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:47.571612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e5a0c-8032-44c9-8743-a6fdc74c2fed admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:47.594588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1131/2281] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:47.595899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf3f03-a94d-49f9-9aa8-dd8eaad35ccc admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:47.611878 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf3f03-a94d-49f9-9aa8-dd8eaad35ccc admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:47.626397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df0cff-2d6e-469e-bdd9-3161d436c132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:47.627002 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df0cff-2d6e-469e-bdd9-3161d436c132 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:47.627276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df0cff-2d6e-469e-bdd9-3161d436c132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:47.628207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df0cff-2d6e-469e-bdd9-3161d436c132 admin admin] PATH_INFO: `/v3/projects/2c73368db1ec4d63907cdad9f58b4225/users/a4ad2711dfc84247a2cad2766d56254a/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:47.640867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:47.666993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df0cff-2d6e-469e-bdd9-3161d436c132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c73368db1ec4d63907cdad9f58b4225, user_id=a4ad2711dfc84247a2cad2766d56254a, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:47.670013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df0cff-2d6e-469e-bdd9-3161d436c132 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:47.699836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1151/2282] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:47 2022] PUT /identity/v3/projects/2c73368db1ec4d63907cdad9f58b4225/users/a4ad2711dfc84247a2cad2766d56254a/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:47.709798 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7a29f-2fa2-4294-b509-737ab3c258bc None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:47.710349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7a29f-2fa2-4294-b509-737ab3c258bc None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:47.710785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7a29f-2fa2-4294-b509-737ab3c258bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:47.782509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-7cc7a29f-2fa2-4294-b509-737ab3c258bc None None] MFA Rules not processed for user `a4ad2711dfc84247a2cad2766d56254a`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:47.784034 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7cc7a29f-2fa2-4294-b509-737ab3c258bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:47.973710 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1132/2283] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:47.974887 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df0cff-2d6e-469e-bdd9-3161d436c132 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:48.081470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df0cff-2d6e-469e-bdd9-3161d436c132 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:48.095240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bba7d-7468-4267-8215-afe682679e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:48.097895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bba7d-7468-4267-8215-afe682679e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:48.098290 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bba7d-7468-4267-8215-afe682679e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:48.098452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bba7d-7468-4267-8215-afe682679e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:48.100798 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bba7d-7468-4267-8215-afe682679e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:48.101393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bba7d-7468-4267-8215-afe682679e70 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:48.177013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1152/2284] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:30:47 2022] POST /identity/v3/projects => generated 377 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:48.178351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7a29f-2fa2-4294-b509-737ab3c258bc None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:48.200915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7a29f-2fa2-4294-b509-737ab3c258bc None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:48.334239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7e4c1-957f-4094-a060-c9efd103dc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:48.334857 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7e4c1-957f-4094-a060-c9efd103dc91 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:48.335137 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7e4c1-957f-4094-a060-c9efd103dc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:48.335416 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7e4c1-957f-4094-a060-c9efd103dc91 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:48.336089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7e4c1-957f-4094-a060-c9efd103dc91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:48.338010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7e4c1-957f-4094-a060-c9efd103dc91 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:48.374835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1133/2285] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:30:48 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:48.376083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bba7d-7468-4267-8215-afe682679e70 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:48.390068 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bba7d-7468-4267-8215-afe682679e70 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:48.421102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35d3a9-4839-4d90-9419-7d34c4ee4ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:48.421735 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35d3a9-4839-4d90-9419-7d34c4ee4ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:48.422088 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35d3a9-4839-4d90-9419-7d34c4ee4ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:48.422336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35d3a9-4839-4d90-9419-7d34c4ee4ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:48.423056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35d3a9-4839-4d90-9419-7d34c4ee4ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:48.425838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35d3a9-4839-4d90-9419-7d34c4ee4ce1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:48.459915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1153/2286] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:48.468182 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7e4c1-957f-4094-a060-c9efd103dc91 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:48.491519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7e4c1-957f-4094-a060-c9efd103dc91 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:48.505520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:48.541881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b517e5-983d-497b-ba47-b575717544f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:48.543631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b517e5-983d-497b-ba47-b575717544f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:48.543938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b517e5-983d-497b-ba47-b575717544f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:48.544520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b517e5-983d-497b-ba47-b575717544f9 admin admin] PATH_INFO: `/v3/projects/e6dcc801ba6f406e8fd0d7792a469a6a/users/30c962179195409e9b6d223f6df1ec4a/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:48.593483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b517e5-983d-497b-ba47-b575717544f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6dcc801ba6f406e8fd0d7792a469a6a, user_id=30c962179195409e9b6d223f6df1ec4a, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:48.596936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b517e5-983d-497b-ba47-b575717544f9 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:48.652645 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1134/2287] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:48 2022] PUT /identity/v3/projects/e6dcc801ba6f406e8fd0d7792a469a6a/users/30c962179195409e9b6d223f6df1ec4a/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:48.656986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35d3a9-4839-4d90-9419-7d34c4ee4ce1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:48.688511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35d3a9-4839-4d90-9419-7d34c4ee4ce1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:48.863395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d28ec-5479-4136-9264-a22e8a762913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:48.880400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d28ec-5479-4136-9264-a22e8a762913 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:48.880868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d28ec-5479-4136-9264-a22e8a762913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:48.881161 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d28ec-5479-4136-9264-a22e8a762913 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:48.883252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d28ec-5479-4136-9264-a22e8a762913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:48.889567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d28ec-5479-4136-9264-a22e8a762913 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:48.917178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1154/2288] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:48.922756 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b517e5-983d-497b-ba47-b575717544f9 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:48.971557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b517e5-983d-497b-ba47-b575717544f9 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:48.987982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e42233-e972-4a82-9c47-d8eab319cfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:48.988718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e42233-e972-4a82-9c47-d8eab319cfba admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:48.990920 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e42233-e972-4a82-9c47-d8eab319cfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:48.991225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e42233-e972-4a82-9c47-d8eab319cfba admin admin] PATH_INFO: `/v3/projects/e6dcc801ba6f406e8fd0d7792a469a6a/users/30c962179195409e9b6d223f6df1ec4a/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:49.000185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e42233-e972-4a82-9c47-d8eab319cfba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6dcc801ba6f406e8fd0d7792a469a6a, user_id=30c962179195409e9b6d223f6df1ec4a, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:49.003538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e42233-e972-4a82-9c47-d8eab319cfba admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:49.036023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1135/2289] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:48 2022] PUT /identity/v3/projects/e6dcc801ba6f406e8fd0d7792a469a6a/users/30c962179195409e9b6d223f6df1ec4a/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:49.045097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8566a8d-d974-43db-8a20-c29e684d0208 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:49.045409 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8566a8d-d974-43db-8a20-c29e684d0208 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:49.045755 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8566a8d-d974-43db-8a20-c29e684d0208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:49.122313 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b8566a8d-d974-43db-8a20-c29e684d0208 None None] MFA Rules not processed for user `30c962179195409e9b6d223f6df1ec4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:49.123554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b8566a8d-d974-43db-8a20-c29e684d0208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:49.262337 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:49.331643 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1155/2290] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:49.361494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e42233-e972-4a82-9c47-d8eab319cfba admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:49.364806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-f2e42233-e972-4a82-9c47-d8eab319cfba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:49.513495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e42233-e972-4a82-9c47-d8eab319cfba admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:49.526883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c48282-90cf-45a2-a2e1-0b1801b98e77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:49.527650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c48282-90cf-45a2-a2e1-0b1801b98e77 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:49.528070 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c48282-90cf-45a2-a2e1-0b1801b98e77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:49.528352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c48282-90cf-45a2-a2e1-0b1801b98e77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:49.704441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c48282-90cf-45a2-a2e1-0b1801b98e77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:49.706752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c48282-90cf-45a2-a2e1-0b1801b98e77 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:49.746437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1136/2291] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:50.379867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf8373-73aa-4a4c-9f33-da302c1998c3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:50.381287 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf8373-73aa-4a4c-9f33-da302c1998c3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:50.381675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf8373-73aa-4a4c-9f33-da302c1998c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:50.432192 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-fdaf8373-73aa-4a4c-9f33-da302c1998c3 None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:50.434226 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fdaf8373-73aa-4a4c-9f33-da302c1998c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:50.562328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1156/2292] 10.222.0.27 () {58 vars in 967 bytes} [Mon Sep 12 23:30:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:50.599174 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c48282-90cf-45a2-a2e1-0b1801b98e77 service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:50.600785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-73c48282-90cf-45a2-a2e1-0b1801b98e77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:50.831640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c48282-90cf-45a2-a2e1-0b1801b98e77 service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:50.845404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb5de0-7fb5-451a-a7d0-ec4153f5b889 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:50.847443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb5de0-7fb5-451a-a7d0-ec4153f5b889 service swift] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:50.848231 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb5de0-7fb5-451a-a7d0-ec4153f5b889 service swift] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:50.849969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb5de0-7fb5-451a-a7d0-ec4153f5b889 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:50.875433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb5de0-7fb5-451a-a7d0-ec4153f5b889 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:50.878429 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb5de0-7fb5-451a-a7d0-ec4153f5b889 service swift] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:50.901886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1137/2293] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:30:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:30:51.890987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf8373-73aa-4a4c-9f33-da302c1998c3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:51.939423 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf8373-73aa-4a4c-9f33-da302c1998c3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:51.945528 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2ea81-3945-415a-98bd-195d553be6bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:51.946662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2ea81-3945-415a-98bd-195d553be6bf service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:51.947124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2ea81-3945-415a-98bd-195d553be6bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:51.947701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2ea81-3945-415a-98bd-195d553be6bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:52.106926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2ea81-3945-415a-98bd-195d553be6bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:52.110100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2ea81-3945-415a-98bd-195d553be6bf service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:52.134839 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1157/2294] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:52.933744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb5de0-7fb5-451a-a7d0-ec4153f5b889 service swift] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:52.945505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb5de0-7fb5-451a-a7d0-ec4153f5b889 service swift] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:52.952191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448e936-d056-47f9-b861-4c53723e75b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:52.952805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448e936-d056-47f9-b861-4c53723e75b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:52.953089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448e936-d056-47f9-b861-4c53723e75b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:52.953366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448e936-d056-47f9-b861-4c53723e75b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:52.964694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448e936-d056-47f9-b861-4c53723e75b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:52.966160 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448e936-d056-47f9-b861-4c53723e75b9 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:52.970070 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:52.991062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1138/2295] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:54.383500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2ea81-3945-415a-98bd-195d553be6bf service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:54.385854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d0c2ea81-3945-415a-98bd-195d553be6bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:54.472090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2ea81-3945-415a-98bd-195d553be6bf service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:54.476569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd40e7b-9f2d-4ce3-a9e3-1bef0f381f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:54.477192 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd40e7b-9f2d-4ce3-a9e3-1bef0f381f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:54.477471 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd40e7b-9f2d-4ce3-a9e3-1bef0f381f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:54.477744 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd40e7b-9f2d-4ce3-a9e3-1bef0f381f65 admin admin] PATH_INFO: `/v3/users/2dcd3dd65d5841049ed20e36e847bb53` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:54.478851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd40e7b-9f2d-4ce3-a9e3-1bef0f381f65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dcd3dd65d5841049ed20e36e847bb53)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:54.481126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd40e7b-9f2d-4ce3-a9e3-1bef0f381f65 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:54.506007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:54.533402 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-afd40e7b-9f2d-4ce3-a9e3-1bef0f381f65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dcd3dd65d5841049ed20e36e847bb53'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:54.540389 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-afd40e7b-9f2d-4ce3-a9e3-1bef0f381f65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dcd3dd65d5841049ed20e36e847bb53'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:54.560615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-afd40e7b-9f2d-4ce3-a9e3-1bef0f381f65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dcd3dd65d5841049ed20e36e847bb53'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:54.568691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-afd40e7b-9f2d-4ce3-a9e3-1bef0f381f65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dcd3dd65d5841049ed20e36e847bb53'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:54.569393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-afd40e7b-9f2d-4ce3-a9e3-1bef0f381f65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dcd3dd65d5841049ed20e36e847bb53'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:54.576374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1158/2296] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:54 2022] DELETE /identity/v3/users/2dcd3dd65d5841049ed20e36e847bb53 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:54.750875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448e936-d056-47f9-b861-4c53723e75b9 service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:54.762331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448e936-d056-47f9-b861-4c53723e75b9 service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:54.825011 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:54.825672 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:54.825982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:54.826258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd admin admin] PATH_INFO: `/v3/projects/641b6882f873400fbef301fc5ce82391` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:54.827538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=641b6882f873400fbef301fc5ce82391)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:54.829470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:54.894049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '641b6882f873400fbef301fc5ce82391', 'request_id': 'req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:54.899952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '641b6882f873400fbef301fc5ce82391', 'request_id': 'req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:54.909307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '641b6882f873400fbef301fc5ce82391', 'request_id': 'req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:54.932238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd admin admin] The token cache is being invalidate because project 641b6882f873400fbef301fc5ce82391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:54.932545 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:54.933752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-22e9e4b7-9cf6-474c-af61-c30a47dc19cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:54.937580 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1139/2297] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:54 2022] DELETE /identity/v3/projects/641b6882f873400fbef301fc5ce82391 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:54.943667 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb0088-fb3f-4c54-8282-aaa2543b96d5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:54.944233 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb0088-fb3f-4c54-8282-aaa2543b96d5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:54.944559 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb0088-fb3f-4c54-8282-aaa2543b96d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:54.966292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a1eb0088-fb3f-4c54-8282-aaa2543b96d5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:54.967644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a1eb0088-fb3f-4c54-8282-aaa2543b96d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:55.029324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1159/2298] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:55.038039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c30e5d-5a05-474f-981a-46479f18d576 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:55.038149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c30e5d-5a05-474f-981a-46479f18d576 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:55.038426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c30e5d-5a05-474f-981a-46479f18d576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:55.059636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-e7c30e5d-5a05-474f-981a-46479f18d576 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:55.060868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e7c30e5d-5a05-474f-981a-46479f18d576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:55.069517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1140/2299] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:55.073847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb0088-fb3f-4c54-8282-aaa2543b96d5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:55.083951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb0088-fb3f-4c54-8282-aaa2543b96d5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:55.088676 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dc35c-714c-4b4a-a917-84e6719dca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:55.089814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dc35c-714c-4b4a-a917-84e6719dca87 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:55.090091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dc35c-714c-4b4a-a917-84e6719dca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:55.090363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dc35c-714c-4b4a-a917-84e6719dca87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:55.091008 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13dc35c-714c-4b4a-a917-84e6719dca87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:30:55.091308 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13dc35c-714c-4b4a-a917-84e6719dca87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:55.093017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13dc35c-714c-4b4a-a917-84e6719dca87 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:55.106159 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1160/2300] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:30:55.110828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c30e5d-5a05-474f-981a-46479f18d576 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:55.120009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c30e5d-5a05-474f-981a-46479f18d576 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:55.124701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c347b-1d4b-49f3-aebe-b6aaca0bf1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:55.125288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c347b-1d4b-49f3-aebe-b6aaca0bf1df admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:55.125565 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c347b-1d4b-49f3-aebe-b6aaca0bf1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:55.125860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c347b-1d4b-49f3-aebe-b6aaca0bf1df admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:55.126506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c347b-1d4b-49f3-aebe-b6aaca0bf1df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:55.128513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c347b-1d4b-49f3-aebe-b6aaca0bf1df admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:55.157397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1141/2301] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:30:55 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:30:55.161692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dc35c-714c-4b4a-a917-84e6719dca87 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:55.172390 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dc35c-714c-4b4a-a917-84e6719dca87 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:55.228917 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477c822-4d15-4e02-9232-40564c0ee3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:55.229440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477c822-4d15-4e02-9232-40564c0ee3c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:55.230345 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477c822-4d15-4e02-9232-40564c0ee3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:55.230621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477c822-4d15-4e02-9232-40564c0ee3c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:55.231265 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477c822-4d15-4e02-9232-40564c0ee3c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:55.233237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477c822-4d15-4e02-9232-40564c0ee3c5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:55.254478 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1161/2302] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:30:55 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:30:55.258556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c347b-1d4b-49f3-aebe-b6aaca0bf1df admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:55.269042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c347b-1d4b-49f3-aebe-b6aaca0bf1df admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:55.274678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad31365-5c98-466c-9f94-956251a2d138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:55.275328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad31365-5c98-466c-9f94-956251a2d138 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:55.275739 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad31365-5c98-466c-9f94-956251a2d138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:55.276082 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad31365-5c98-466c-9f94-956251a2d138 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:55.277498 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad31365-5c98-466c-9f94-956251a2d138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:55.279515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad31365-5c98-466c-9f94-956251a2d138 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:55.290189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1142/2303] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:55.295453 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477c822-4d15-4e02-9232-40564c0ee3c5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:55.306109 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477c822-4d15-4e02-9232-40564c0ee3c5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:55.310844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2045a3-a054-4480-a7d7-95e0435bf9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:55.311448 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2045a3-a054-4480-a7d7-95e0435bf9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:55.311770 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2045a3-a054-4480-a7d7-95e0435bf9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:55.312050 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2045a3-a054-4480-a7d7-95e0435bf9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:55.312701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2045a3-a054-4480-a7d7-95e0435bf9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:55.314302 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2045a3-a054-4480-a7d7-95e0435bf9c8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:55.328340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1162/2304] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:30:55.329481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad31365-5c98-466c-9f94-956251a2d138 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:55.342969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad31365-5c98-466c-9f94-956251a2d138 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:55.348386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6e08f-4a05-4cc0-bfa5-bf3dde31e611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:55.349045 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e08f-4a05-4cc0-bfa5-bf3dde31e611 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:55.349329 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e08f-4a05-4cc0-bfa5-bf3dde31e611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:55.349606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e08f-4a05-4cc0-bfa5-bf3dde31e611 admin admin] PATH_INFO: `/v3/projects/cc8ed3c153274c21b50152e243f63787/users/e4ed7c3206d54413bd064274528891d6/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:55.366810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6e08f-4a05-4cc0-bfa5-bf3dde31e611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc8ed3c153274c21b50152e243f63787, user_id=e4ed7c3206d54413bd064274528891d6, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:55.369829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6e08f-4a05-4cc0-bfa5-bf3dde31e611 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:55.383887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1143/2305] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:30:55 2022] PUT /identity/v3/projects/cc8ed3c153274c21b50152e243f63787/users/e4ed7c3206d54413bd064274528891d6/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:55.394894 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4b7f5-b062-41ff-b8cb-4f18d73a2719 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:55.395414 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4b7f5-b062-41ff-b8cb-4f18d73a2719 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:55.395542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4b7f5-b062-41ff-b8cb-4f18d73a2719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:55.409767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:55.427417 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c7a4b7f5-b062-41ff-b8cb-4f18d73a2719 None None] MFA Rules not processed for user `e4ed7c3206d54413bd064274528891d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:55.428808 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c7a4b7f5-b062-41ff-b8cb-4f18d73a2719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:55.496120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1163/2306] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:55.507084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6e08f-4a05-4cc0-bfa5-bf3dde31e611 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:55.508833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-76d6e08f-4a05-4cc0-bfa5-bf3dde31e611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:55.539343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:55.571151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6e08f-4a05-4cc0-bfa5-bf3dde31e611 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:55.578631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f0b4e-d573-4a76-b5bb-499fdd42b53a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:55.579261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f0b4e-d573-4a76-b5bb-499fdd42b53a service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:55.579546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f0b4e-d573-4a76-b5bb-499fdd42b53a service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:55.579856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f0b4e-d573-4a76-b5bb-499fdd42b53a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:55.588977 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f0b4e-d573-4a76-b5bb-499fdd42b53a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:55.591060 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f0b4e-d573-4a76-b5bb-499fdd42b53a service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:55.601134 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1144/2307] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:56.991315 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4b7f5-b062-41ff-b8cb-4f18d73a2719 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:56.992883 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c7a4b7f5-b062-41ff-b8cb-4f18d73a2719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:57.060623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4b7f5-b062-41ff-b8cb-4f18d73a2719 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:57.065312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794e1cd-c52f-44be-b7f4-dfb2e08ada6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:57.065937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794e1cd-c52f-44be-b7f4-dfb2e08ada6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:57.066219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794e1cd-c52f-44be-b7f4-dfb2e08ada6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:57.066496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794e1cd-c52f-44be-b7f4-dfb2e08ada6f admin admin] PATH_INFO: `/v3/users/eb1ef4a8d3d74cb29b07fb66c5065011` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:57.067966 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794e1cd-c52f-44be-b7f4-dfb2e08ada6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb1ef4a8d3d74cb29b07fb66c5065011)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:57.070352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794e1cd-c52f-44be-b7f4-dfb2e08ada6f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:57.128303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3794e1cd-c52f-44be-b7f4-dfb2e08ada6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb1ef4a8d3d74cb29b07fb66c5065011'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:57.135649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3794e1cd-c52f-44be-b7f4-dfb2e08ada6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb1ef4a8d3d74cb29b07fb66c5065011'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:57.149778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3794e1cd-c52f-44be-b7f4-dfb2e08ada6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb1ef4a8d3d74cb29b07fb66c5065011'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:57.161339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3794e1cd-c52f-44be-b7f4-dfb2e08ada6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb1ef4a8d3d74cb29b07fb66c5065011'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:57.162042 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3794e1cd-c52f-44be-b7f4-dfb2e08ada6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb1ef4a8d3d74cb29b07fb66c5065011'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:57.168726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1164/2308] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:56 2022] DELETE /identity/v3/users/eb1ef4a8d3d74cb29b07fb66c5065011 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:57.173206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86408a9-02fd-4ef9-a595-393a1f1412bd None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:57.173683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86408a9-02fd-4ef9-a595-393a1f1412bd None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:57.173683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86408a9-02fd-4ef9-a595-393a1f1412bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:57.193097 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-e86408a9-02fd-4ef9-a595-393a1f1412bd None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:57.194386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e86408a9-02fd-4ef9-a595-393a1f1412bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:57.259783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1145/2309] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:57.270461 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794e1cd-c52f-44be-b7f4-dfb2e08ada6f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:57.272029 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3794e1cd-c52f-44be-b7f4-dfb2e08ada6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:57.300707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:57.365167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794e1cd-c52f-44be-b7f4-dfb2e08ada6f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:57.370831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10ef86-8b86-4493-a9cb-1d3fcb6c9e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:57.371751 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10ef86-8b86-4493-a9cb-1d3fcb6c9e95 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:57.371987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10ef86-8b86-4493-a9cb-1d3fcb6c9e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:57.372270 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10ef86-8b86-4493-a9cb-1d3fcb6c9e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:57.419635 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10ef86-8b86-4493-a9cb-1d3fcb6c9e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:57.425048 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10ef86-8b86-4493-a9cb-1d3fcb6c9e95 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:57.457681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1165/2310] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:30:57.875933 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86408a9-02fd-4ef9-a595-393a1f1412bd None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:57.878055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e86408a9-02fd-4ef9-a595-393a1f1412bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:57.901200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86408a9-02fd-4ef9-a595-393a1f1412bd None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:57.906875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12446b86-fb24-47f9-9636-99162480fbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:57.907700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12446b86-fb24-47f9-9636-99162480fbe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:57.908036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12446b86-fb24-47f9-9636-99162480fbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:57.908250 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12446b86-fb24-47f9-9636-99162480fbe7 admin admin] PATH_INFO: `/v3/projects/c947c5f05ce146acb92d80c92ebf1705` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:57.909790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12446b86-fb24-47f9-9636-99162480fbe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c947c5f05ce146acb92d80c92ebf1705)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:57.914094 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12446b86-fb24-47f9-9636-99162480fbe7 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:57.987896 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-12446b86-fb24-47f9-9636-99162480fbe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c947c5f05ce146acb92d80c92ebf1705', 'request_id': 'req-12446b86-fb24-47f9-9636-99162480fbe7'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:57.994256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-12446b86-fb24-47f9-9636-99162480fbe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c947c5f05ce146acb92d80c92ebf1705', 'request_id': 'req-12446b86-fb24-47f9-9636-99162480fbe7'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.009455 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-12446b86-fb24-47f9-9636-99162480fbe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c947c5f05ce146acb92d80c92ebf1705', 'request_id': 'req-12446b86-fb24-47f9-9636-99162480fbe7'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.021654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-12446b86-fb24-47f9-9636-99162480fbe7 admin admin] The token cache is being invalidate because project c947c5f05ce146acb92d80c92ebf1705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:58.021952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-12446b86-fb24-47f9-9636-99162480fbe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.022907 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-12446b86-fb24-47f9-9636-99162480fbe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.024707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1146/2311] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:57 2022] DELETE /identity/v3/projects/c947c5f05ce146acb92d80c92ebf1705 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:58.029606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10ef86-8b86-4493-a9cb-1d3fcb6c9e95 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:58.042677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10ef86-8b86-4493-a9cb-1d3fcb6c9e95 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:58.103451 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d69361-b358-4e51-852c-3dbd4b5cb7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:58.104227 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d69361-b358-4e51-852c-3dbd4b5cb7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:58.105226 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d69361-b358-4e51-852c-3dbd4b5cb7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:58.105570 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d69361-b358-4e51-852c-3dbd4b5cb7d8 admin admin] PATH_INFO: `/v3/users/a4ad2711dfc84247a2cad2766d56254a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:58.106761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d69361-b358-4e51-852c-3dbd4b5cb7d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4ad2711dfc84247a2cad2766d56254a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:58.108757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d69361-b358-4e51-852c-3dbd4b5cb7d8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:58.159963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-51d69361-b358-4e51-852c-3dbd4b5cb7d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4ad2711dfc84247a2cad2766d56254a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.168251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-51d69361-b358-4e51-852c-3dbd4b5cb7d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4ad2711dfc84247a2cad2766d56254a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.181704 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-51d69361-b358-4e51-852c-3dbd4b5cb7d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4ad2711dfc84247a2cad2766d56254a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.190556 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-51d69361-b358-4e51-852c-3dbd4b5cb7d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4ad2711dfc84247a2cad2766d56254a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.191350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-51d69361-b358-4e51-852c-3dbd4b5cb7d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4ad2711dfc84247a2cad2766d56254a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.199685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1166/2312] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:58 2022] DELETE /identity/v3/users/a4ad2711dfc84247a2cad2766d56254a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:58.255342 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191ec0c-9709-4eb5-a3c4-31919a5ceffc None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:58.255990 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191ec0c-9709-4eb5-a3c4-31919a5ceffc None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:58.256230 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191ec0c-9709-4eb5-a3c4-31919a5ceffc None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:58.258062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1147/2313] 10.222.0.27 () {54 vars in 921 bytes} [Mon Sep 12 23:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:30:58.266211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8842088-fc38-4727-8070-1f583eee2225 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:58.266751 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8842088-fc38-4727-8070-1f583eee2225 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:58.267039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8842088-fc38-4727-8070-1f583eee2225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:58.288931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d8842088-fc38-4727-8070-1f583eee2225 None None] MFA Rules not processed for user `f5fa28ffdafd435cb2d93bd453add3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:58.290217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d8842088-fc38-4727-8070-1f583eee2225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:58.331903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:58.393763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1167/2314] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Sep 12 23:30:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:58.398059 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191ec0c-9709-4eb5-a3c4-31919a5ceffc None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:58.399850 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1191ec0c-9709-4eb5-a3c4-31919a5ceffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:58.484801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191ec0c-9709-4eb5-a3c4-31919a5ceffc None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:58.492531 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe36e25-d4c3-4f0e-96ad-ca3a1c662634 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61407406ef242a78030f79532fe2247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:58.494150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe36e25-d4c3-4f0e-96ad-ca3a1c662634 service glance] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:58.494897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe36e25-d4c3-4f0e-96ad-ca3a1c662634 service glance] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:58.495856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe36e25-d4c3-4f0e-96ad-ca3a1c662634 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:58.504806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:58.509582 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe36e25-d4c3-4f0e-96ad-ca3a1c662634 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:58.511883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe36e25-d4c3-4f0e-96ad-ca3a1c662634 service glance] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:58.525196 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1148/2315] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:58.545620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8842088-fc38-4727-8070-1f583eee2225 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:58.548516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d8842088-fc38-4727-8070-1f583eee2225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:58.647022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6f73b-8aca-41cb-b8b2-f9348aa6fa41 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:58.647612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6f73b-8aca-41cb-b8b2-f9348aa6fa41 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:58.647984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6f73b-8aca-41cb-b8b2-f9348aa6fa41 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:58.649725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1149/2316] 10.222.0.27 () {54 vars in 921 bytes} [Mon Sep 12 23:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:30:58.658500 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710e7c9-583f-421c-8acf-021f0df3a182 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:58.659126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710e7c9-583f-421c-8acf-021f0df3a182 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:58.659419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710e7c9-583f-421c-8acf-021f0df3a182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:58.661939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8842088-fc38-4727-8070-1f583eee2225 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:58.667652 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:58.668305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:58.668633 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:58.668875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] PATH_INFO: `/v3/projects/2c73368db1ec4d63907cdad9f58b4225` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:58.670125 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c73368db1ec4d63907cdad9f58b4225)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:58.676228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:58.684885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-d710e7c9-583f-421c-8acf-021f0df3a182 None None] MFA Rules not processed for user `bc6e63e9fc3447d5a894a777a98fe773`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:58.686195 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d710e7c9-583f-421c-8acf-021f0df3a182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:58.742474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c73368db1ec4d63907cdad9f58b4225', 'request_id': 'req-7704da0e-c36d-456d-8191-24b343594a5f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.755901 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c73368db1ec4d63907cdad9f58b4225', 'request_id': 'req-7704da0e-c36d-456d-8191-24b343594a5f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.765837 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1150/2317] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Sep 12 23:30:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:58.766207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c73368db1ec4d63907cdad9f58b4225', 'request_id': 'req-7704da0e-c36d-456d-8191-24b343594a5f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.779696 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710e7c9-583f-421c-8acf-021f0df3a182 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:58.780326 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] The token cache is being invalidate because project 2c73368db1ec4d63907cdad9f58b4225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:58.780622 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.781312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.783353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1168/2318] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:58 2022] DELETE /identity/v3/projects/2c73368db1ec4d63907cdad9f58b4225 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:58.784894 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d710e7c9-583f-421c-8acf-021f0df3a182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:58.787271 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:58.788750 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:58.799006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710e7c9-583f-421c-8acf-021f0df3a182 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:58.877876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6f097-3bb0-4631-8207-de19032a10a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:58.878611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f097-3bb0-4631-8207-de19032a10a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:58.878891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f097-3bb0-4631-8207-de19032a10a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:58.879252 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f097-3bb0-4631-8207-de19032a10a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:58.881345 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b5d6f097-3bb0-4631-8207-de19032a10a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:58.893932 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704da0e-c36d-456d-8191-24b343594a5f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:58.902019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac149a-f863-466f-a0fe-be1b45b89d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:58.902778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac149a-f863-466f-a0fe-be1b45b89d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:58.903068 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac149a-f863-466f-a0fe-be1b45b89d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:58.903341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac149a-f863-466f-a0fe-be1b45b89d48 admin admin] PATH_INFO: `/v3/users/30c962179195409e9b6d223f6df1ec4a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:58.905158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac149a-f863-466f-a0fe-be1b45b89d48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30c962179195409e9b6d223f6df1ec4a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:58.908136 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac149a-f863-466f-a0fe-be1b45b89d48 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:58.909308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6f097-3bb0-4631-8207-de19032a10a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:58.910088 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:58.911906 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6f097-3bb0-4631-8207-de19032a10a9 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:58.929669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1151/2319] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:30:58.973327 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f1ac149a-f863-466f-a0fe-be1b45b89d48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30c962179195409e9b6d223f6df1ec4a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:58.994892 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f1ac149a-f863-466f-a0fe-be1b45b89d48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30c962179195409e9b6d223f6df1ec4a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:59.012553 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f1ac149a-f863-466f-a0fe-be1b45b89d48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30c962179195409e9b6d223f6df1ec4a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:59.025138 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f1ac149a-f863-466f-a0fe-be1b45b89d48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30c962179195409e9b6d223f6df1ec4a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:59.026210 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f1ac149a-f863-466f-a0fe-be1b45b89d48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30c962179195409e9b6d223f6df1ec4a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:59.035530 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1169/2320] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:30:58 2022] DELETE /identity/v3/users/30c962179195409e9b6d223f6df1ec4a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:30:59.041570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b435573-16ff-45bd-a99b-d750f5ae8ed7 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:59.041857 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b435573-16ff-45bd-a99b-d750f5ae8ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:59.042236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b435573-16ff-45bd-a99b-d750f5ae8ed7 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:59.043761 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1152/2321] 10.222.0.27 () {54 vars in 921 bytes} [Mon Sep 12 23:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:30:59.049880 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ddd35-1683-4163-80d5-f94d9e48ba82 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:59.050439 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ddd35-1683-4163-80d5-f94d9e48ba82 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:59.050833 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ddd35-1683-4163-80d5-f94d9e48ba82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:59.091180 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-770ddd35-1683-4163-80d5-f94d9e48ba82 None None] MFA Rules not processed for user `f5fa28ffdafd435cb2d93bd453add3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:30:59.092760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-770ddd35-1683-4163-80d5-f94d9e48ba82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:59.176505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1170/2322] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Sep 12 23:30:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:30:59.188849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b435573-16ff-45bd-a99b-d750f5ae8ed7 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:59.190568 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3b435573-16ff-45bd-a99b-d750f5ae8ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:59.252274 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b435573-16ff-45bd-a99b-d750f5ae8ed7 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:59.255845 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40e8aa-b2c5-49dc-a112-7b35c55dc772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:59.256433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40e8aa-b2c5-49dc-a112-7b35c55dc772 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:59.256711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40e8aa-b2c5-49dc-a112-7b35c55dc772 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:59.256979 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40e8aa-b2c5-49dc-a112-7b35c55dc772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:59.269085 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40e8aa-b2c5-49dc-a112-7b35c55dc772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:59.270860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40e8aa-b2c5-49dc-a112-7b35c55dc772 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:59.283162 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1153/2323] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:30:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:30:59.374801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ddd35-1683-4163-80d5-f94d9e48ba82 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:30:59.376725 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-770ddd35-1683-4163-80d5-f94d9e48ba82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:30:59.466796 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ddd35-1683-4163-80d5-f94d9e48ba82 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:30:59.472651 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:30:59.473610 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:30:59.474145 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:30:59.474447 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] PATH_INFO: `/v3/projects/e6dcc801ba6f406e8fd0d7792a469a6a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:30:59.475975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6dcc801ba6f406e8fd0d7792a469a6a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:30:59.478343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:30:59.491822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:30:59.539094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6dcc801ba6f406e8fd0d7792a469a6a', 'request_id': 'req-1cfc0701-4706-4b7a-8b03-bed8bff1a518'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:59.546203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6dcc801ba6f406e8fd0d7792a469a6a', 'request_id': 'req-1cfc0701-4706-4b7a-8b03-bed8bff1a518'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:59.560499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6dcc801ba6f406e8fd0d7792a469a6a', 'request_id': 'req-1cfc0701-4706-4b7a-8b03-bed8bff1a518'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:59.574480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] The token cache is being invalidate because project e6dcc801ba6f406e8fd0d7792a469a6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:30:59.574771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:59.575436 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:30:59.577300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1171/2324] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:30:59 2022] DELETE /identity/v3/projects/e6dcc801ba6f406e8fd0d7792a469a6a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:31:02.027208 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b5dbe-13c0-4f4a-855c-88895bcb6eb7 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:02.027709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b5dbe-13c0-4f4a-855c-88895bcb6eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:02.028289 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b5dbe-13c0-4f4a-855c-88895bcb6eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:02.068662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-5c5b5dbe-13c0-4f4a-855c-88895bcb6eb7 None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:31:02.072786 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5c5b5dbe-13c0-4f4a-855c-88895bcb6eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:02.171740 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:31:02.185076 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1154/2325] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:31:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:31:02.197958 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:02.201564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:02.290407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc0701-4706-4b7a-8b03-bed8bff1a518 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:02.296702 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1f19a-0f99-4eaf-bf1e-71df7ac750a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:02.297747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1f19a-0f99-4eaf-bf1e-71df7ac750a2 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:02.298033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1f19a-0f99-4eaf-bf1e-71df7ac750a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:02.298313 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1f19a-0f99-4eaf-bf1e-71df7ac750a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:02.313981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1f19a-0f99-4eaf-bf1e-71df7ac750a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:02.316103 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1f19a-0f99-4eaf-bf1e-71df7ac750a2 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:02.326285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1172/2326] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:31:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:31:06.520740 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b5dbe-13c0-4f4a-855c-88895bcb6eb7 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:06.522344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5c5b5dbe-13c0-4f4a-855c-88895bcb6eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:06.596998 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b5dbe-13c0-4f4a-855c-88895bcb6eb7 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:06.601633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee116e91-3a30-4d3f-807c-725ba5380136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:06.602293 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee116e91-3a30-4d3f-807c-725ba5380136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:06.602577 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee116e91-3a30-4d3f-807c-725ba5380136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:06.602854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee116e91-3a30-4d3f-807c-725ba5380136 admin admin] PATH_INFO: `/v3/users/e4ed7c3206d54413bd064274528891d6` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:06.604317 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee116e91-3a30-4d3f-807c-725ba5380136 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4ed7c3206d54413bd064274528891d6)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:06.606364 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee116e91-3a30-4d3f-807c-725ba5380136 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:06.658876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-ee116e91-3a30-4d3f-807c-725ba5380136 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4ed7c3206d54413bd064274528891d6'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:31:06.674574 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-ee116e91-3a30-4d3f-807c-725ba5380136 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4ed7c3206d54413bd064274528891d6'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:31:06.683832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-ee116e91-3a30-4d3f-807c-725ba5380136 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4ed7c3206d54413bd064274528891d6'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:31:06.684685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-ee116e91-3a30-4d3f-807c-725ba5380136 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4ed7c3206d54413bd064274528891d6'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:31:06.694888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-ee116e91-3a30-4d3f-807c-725ba5380136 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4ed7c3206d54413bd064274528891d6'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:31:06.702574 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1155/2327] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:31:06 2022] DELETE /identity/v3/users/e4ed7c3206d54413bd064274528891d6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:31:06.708914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92829588-159b-429f-a22c-ec31b85fc01c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:06.709477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92829588-159b-429f-a22c-ec31b85fc01c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:06.709477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92829588-159b-429f-a22c-ec31b85fc01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:06.732813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-92829588-159b-429f-a22c-ec31b85fc01c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:31:06.734504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-92829588-159b-429f-a22c-ec31b85fc01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:06.832711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1173/2328] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:31:06.842947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee116e91-3a30-4d3f-807c-725ba5380136 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:06.844529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ee116e91-3a30-4d3f-807c-725ba5380136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:06.935619 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee116e91-3a30-4d3f-807c-725ba5380136 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:06.953908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e443ef-42a5-49f7-82ff-b21382a0cad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:06.954724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e443ef-42a5-49f7-82ff-b21382a0cad5 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:06.955603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e443ef-42a5-49f7-82ff-b21382a0cad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:06.955603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e443ef-42a5-49f7-82ff-b21382a0cad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:06.967634 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e443ef-42a5-49f7-82ff-b21382a0cad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:06.971935 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e443ef-42a5-49f7-82ff-b21382a0cad5 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:07.003801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1156/2329] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:31:07.421996 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92829588-159b-429f-a22c-ec31b85fc01c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:07.423528 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-92829588-159b-429f-a22c-ec31b85fc01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:07.437169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92829588-159b-429f-a22c-ec31b85fc01c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:07.456131 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:07.456811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:07.457096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:07.457375 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36 admin admin] PATH_INFO: `/v3/projects/cc8ed3c153274c21b50152e243f63787` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:07.459764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc8ed3c153274c21b50152e243f63787)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:07.472396 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:07.523920 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:31:07.543539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc8ed3c153274c21b50152e243f63787', 'request_id': 'req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:31:07.549083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc8ed3c153274c21b50152e243f63787', 'request_id': 'req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:31:07.556447 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc8ed3c153274c21b50152e243f63787', 'request_id': 'req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:31:07.568558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36 admin admin] The token cache is being invalidate because project cc8ed3c153274c21b50152e243f63787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:31:07.568844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:31:07.569498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c3a9e035-876c-4cb8-85a0-7c5f8c803d36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:31:07.574929 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1174/2330] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:31:07 2022] DELETE /identity/v3/projects/cc8ed3c153274c21b50152e243f63787 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:31:07.577356 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa6b97-376c-49ec-9555-a72ff812afd9 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:07.577953 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa6b97-376c-49ec-9555-a72ff812afd9 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:07.578052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa6b97-376c-49ec-9555-a72ff812afd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:07.599786 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-63fa6b97-376c-49ec-9555-a72ff812afd9 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:31:07.601408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-63fa6b97-376c-49ec-9555-a72ff812afd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:07.611836 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:31:07.681214 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1157/2331] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:31:07.689200 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a4d8e-b8a3-44ca-ba4d-60f05be606bd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:07.689483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a4d8e-b8a3-44ca-ba4d-60f05be606bd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:07.689759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a4d8e-b8a3-44ca-ba4d-60f05be606bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:07.710876 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3c8a4d8e-b8a3-44ca-ba4d-60f05be606bd None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:31:07.712115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3c8a4d8e-b8a3-44ca-ba4d-60f05be606bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:07.721475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1175/2332] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:31:07.725905 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa6b97-376c-49ec-9555-a72ff812afd9 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:07.737521 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa6b97-376c-49ec-9555-a72ff812afd9 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:07.742721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2781360-3f82-4650-9d86-e1106a32afff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:07.743371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2781360-3f82-4650-9d86-e1106a32afff admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:07.743669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2781360-3f82-4650-9d86-e1106a32afff admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:07.743964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2781360-3f82-4650-9d86-e1106a32afff admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:07.744747 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2781360-3f82-4650-9d86-e1106a32afff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:31:07.745061 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2781360-3f82-4650-9d86-e1106a32afff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:07.746869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2781360-3f82-4650-9d86-e1106a32afff admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:07.759847 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1158/2333] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:31:07.764860 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a4d8e-b8a3-44ca-ba4d-60f05be606bd None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:07.783022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a4d8e-b8a3-44ca-ba4d-60f05be606bd None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:07.805812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b013b1-c7c5-41a1-890f-368c24178f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:07.805812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b013b1-c7c5-41a1-890f-368c24178f72 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:07.805812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b013b1-c7c5-41a1-890f-368c24178f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:07.805812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b013b1-c7c5-41a1-890f-368c24178f72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:07.806389 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b013b1-c7c5-41a1-890f-368c24178f72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:07.808701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b013b1-c7c5-41a1-890f-368c24178f72 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:07.925360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1176/2334] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:31:07 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:31:07.929535 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2781360-3f82-4650-9d86-e1106a32afff admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:07.941007 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2781360-3f82-4650-9d86-e1106a32afff admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:08.016896 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3721d-8672-4f70-840b-25c3d327f33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:08.017704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3721d-8672-4f70-840b-25c3d327f33d admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:08.022176 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3721d-8672-4f70-840b-25c3d327f33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:08.022541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3721d-8672-4f70-840b-25c3d327f33d admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:08.023324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3721d-8672-4f70-840b-25c3d327f33d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:08.025578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3721d-8672-4f70-840b-25c3d327f33d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:08.066301 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1159/2335] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:31:07 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:31:08.067546 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b013b1-c7c5-41a1-890f-368c24178f72 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:08.087741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b013b1-c7c5-41a1-890f-368c24178f72 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:08.093490 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62a653-695f-4c50-8c7a-bcbbce211081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:08.094158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62a653-695f-4c50-8c7a-bcbbce211081 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:08.094523 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62a653-695f-4c50-8c7a-bcbbce211081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:08.094850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62a653-695f-4c50-8c7a-bcbbce211081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:08.095609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62a653-695f-4c50-8c7a-bcbbce211081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:08.097524 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62a653-695f-4c50-8c7a-bcbbce211081 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:08.113929 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1177/2336] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:31:08.124023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3721d-8672-4f70-840b-25c3d327f33d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:08.141754 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3721d-8672-4f70-840b-25c3d327f33d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:08.146555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e699d75-c1c6-4e50-b9fc-27f934fc739f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:08.147173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e699d75-c1c6-4e50-b9fc-27f934fc739f admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:08.147455 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e699d75-c1c6-4e50-b9fc-27f934fc739f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:08.147764 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e699d75-c1c6-4e50-b9fc-27f934fc739f admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:08.148414 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e699d75-c1c6-4e50-b9fc-27f934fc739f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:08.164811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e699d75-c1c6-4e50-b9fc-27f934fc739f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:08.204359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1160/2337] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:31:08.209759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62a653-695f-4c50-8c7a-bcbbce211081 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:08.225738 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62a653-695f-4c50-8c7a-bcbbce211081 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:08.239706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e34c68-bebf-4492-87d8-2cfb70010a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:08.240795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e34c68-bebf-4492-87d8-2cfb70010a4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:08.241353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e34c68-bebf-4492-87d8-2cfb70010a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:08.243303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e34c68-bebf-4492-87d8-2cfb70010a4a admin admin] PATH_INFO: `/v3/projects/e3ac3aed7b1e495f81dc29d224be1eba/users/2e22d2226f77454fb3f155dbf3fdf471/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:08.301122 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e34c68-bebf-4492-87d8-2cfb70010a4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3ac3aed7b1e495f81dc29d224be1eba, user_id=2e22d2226f77454fb3f155dbf3fdf471, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:08.303973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e34c68-bebf-4492-87d8-2cfb70010a4a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:08.327684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1178/2338] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:31:08 2022] PUT /identity/v3/projects/e3ac3aed7b1e495f81dc29d224be1eba/users/2e22d2226f77454fb3f155dbf3fdf471/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:31:08.332556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25a6c9-915b-44f8-8414-af79a112aa08 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:08.332942 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25a6c9-915b-44f8-8414-af79a112aa08 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:08.333219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25a6c9-915b-44f8-8414-af79a112aa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:08.368295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-cf25a6c9-915b-44f8-8414-af79a112aa08 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:31:08.368295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-cf25a6c9-915b-44f8-8414-af79a112aa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:08.370301 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901272e-0678-4e17-8061-66a0d4989bba None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:08.370584 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901272e-0678-4e17-8061-66a0d4989bba None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:08.370858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901272e-0678-4e17-8061-66a0d4989bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:08.400243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d901272e-0678-4e17-8061-66a0d4989bba None None] MFA Rules not processed for user `2d62232282dc4455834857d5290eb169`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:31:08.400544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d901272e-0678-4e17-8061-66a0d4989bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:08.462462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1161/2339] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:31:08.472656 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25a6c9-915b-44f8-8414-af79a112aa08 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:08.474707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-cf25a6c9-915b-44f8-8414-af79a112aa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:08.505369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1179/2340] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:31:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:31:08.523949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:31:08.604762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25a6c9-915b-44f8-8414-af79a112aa08 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:08.609407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eab8bd-f930-4f1e-993a-741a12e1b672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:08.610132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eab8bd-f930-4f1e-993a-741a12e1b672 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:08.610480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eab8bd-f930-4f1e-993a-741a12e1b672 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:08.615191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eab8bd-f930-4f1e-993a-741a12e1b672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:08.633856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eab8bd-f930-4f1e-993a-741a12e1b672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:08.638282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eab8bd-f930-4f1e-993a-741a12e1b672 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:08.652344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1162/2341] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:31:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:31:13.050037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14750b56-859c-4d54-acd3-14c9c81e0fb1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:13.050692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14750b56-859c-4d54-acd3-14c9c81e0fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:13.050788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14750b56-859c-4d54-acd3-14c9c81e0fb1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:13.054703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1180/2342] 10.222.0.27 () {54 vars in 961 bytes} [Mon Sep 12 23:31:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:31:13.061171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318f8a6-3799-4f28-863d-321edd2de96e None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:13.061736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318f8a6-3799-4f28-863d-321edd2de96e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:13.061736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318f8a6-3799-4f28-863d-321edd2de96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:13.097576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-7318f8a6-3799-4f28-863d-321edd2de96e None None] MFA Rules not processed for user `2b3b0a0317b14e29a32eb6ba6b89c2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:31:13.101053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7318f8a6-3799-4f28-863d-321edd2de96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:13.209577 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1163/2343] 10.222.0.27 () {58 vars in 1058 bytes} [Mon Sep 12 23:31:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:31:13.218805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14750b56-859c-4d54-acd3-14c9c81e0fb1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:13.237330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14750b56-859c-4d54-acd3-14c9c81e0fb1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:13.243804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2810b-aed6-4d43-a5e4-06dce7c49221 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b3b0a0317b14e29a32eb6ba6b89c2aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:13.245234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2810b-aed6-4d43-a5e4-06dce7c49221 service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:13.245742 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2810b-aed6-4d43-a5e4-06dce7c49221 service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:13.246343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2810b-aed6-4d43-a5e4-06dce7c49221 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:13.248696 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4fe2810b-aed6-4d43-a5e4-06dce7c49221 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:13.286785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2810b-aed6-4d43-a5e4-06dce7c49221 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:13.295769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2810b-aed6-4d43-a5e4-06dce7c49221 service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:13.330541 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1181/2344] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:31:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:31:14.704570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf70a6-81d3-4bde-a8d2-1863d794aa88 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:14.705039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf70a6-81d3-4bde-a8d2-1863d794aa88 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:14.705130 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf70a6-81d3-4bde-a8d2-1863d794aa88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:14.770716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-fcbf70a6-81d3-4bde-a8d2-1863d794aa88 None None] MFA Rules not processed for user `2e22d2226f77454fb3f155dbf3fdf471`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:31:14.772018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-fcbf70a6-81d3-4bde-a8d2-1863d794aa88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:14.905046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1164/2345] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:31:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:31:14.906412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2810b-aed6-4d43-a5e4-06dce7c49221 service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:14.925224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2810b-aed6-4d43-a5e4-06dce7c49221 service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:14.940028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b2811-daa1-49f9-96c1-a93905203217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:14.942681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b2811-daa1-49f9-96c1-a93905203217 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:14.944894 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b2811-daa1-49f9-96c1-a93905203217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:14.945255 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b2811-daa1-49f9-96c1-a93905203217 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:14.945994 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b2811-daa1-49f9-96c1-a93905203217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:14.951664 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b2811-daa1-49f9-96c1-a93905203217 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:14.998744 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1182/2346] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:31:14 2022] POST /identity/v3/projects => generated 369 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:31:15.004582 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf70a6-81d3-4bde-a8d2-1863d794aa88 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:15.019677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf70a6-81d3-4bde-a8d2-1863d794aa88 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:15.075679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:31:15.089140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91309943-15ed-40d7-87c9-19f91ec1ee34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:15.089919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91309943-15ed-40d7-87c9-19f91ec1ee34 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:15.090205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91309943-15ed-40d7-87c9-19f91ec1ee34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:15.090477 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91309943-15ed-40d7-87c9-19f91ec1ee34 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:15.091873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91309943-15ed-40d7-87c9-19f91ec1ee34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:15.094398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91309943-15ed-40d7-87c9-19f91ec1ee34 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:15.124521 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1165/2347] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:31:15 2022] POST /identity/v3/users => generated 291 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:31:15.125538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b2811-daa1-49f9-96c1-a93905203217 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:15.141932 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b2811-daa1-49f9-96c1-a93905203217 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:15.153796 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b4f44f-8cf4-4b41-b2b5-617a4641986b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:15.157680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4f44f-8cf4-4b41-b2b5-617a4641986b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:15.160224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4f44f-8cf4-4b41-b2b5-617a4641986b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:15.160751 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4f44f-8cf4-4b41-b2b5-617a4641986b admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:15.161686 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b4f44f-8cf4-4b41-b2b5-617a4641986b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:15.164278 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b4f44f-8cf4-4b41-b2b5-617a4641986b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:15.179343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1183/2348] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:31:15.184737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91309943-15ed-40d7-87c9-19f91ec1ee34 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:15.195012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91309943-15ed-40d7-87c9-19f91ec1ee34 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:15.199771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46902dfe-b3ca-4770-8447-c8237b0ac988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:15.202895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46902dfe-b3ca-4770-8447-c8237b0ac988 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:15.202895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46902dfe-b3ca-4770-8447-c8237b0ac988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:15.202895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46902dfe-b3ca-4770-8447-c8237b0ac988 admin admin] PATH_INFO: `/v3/projects/13f4b2186b7342c28a9b62c740750db6/users/796ccf5ac7fc429db3d181208c72f1c9/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:15.222590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46902dfe-b3ca-4770-8447-c8237b0ac988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f4b2186b7342c28a9b62c740750db6, user_id=796ccf5ac7fc429db3d181208c72f1c9, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:15.225705 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46902dfe-b3ca-4770-8447-c8237b0ac988 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:15.242771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1166/2349] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:31:15 2022] PUT /identity/v3/projects/13f4b2186b7342c28a9b62c740750db6/users/796ccf5ac7fc429db3d181208c72f1c9/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:31:15.247887 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b4f44f-8cf4-4b41-b2b5-617a4641986b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:15.258562 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b4f44f-8cf4-4b41-b2b5-617a4641986b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:15.264064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:31:15.334166 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da155ff-3d17-417d-b675-b35ecbd3d177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:15.335216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da155ff-3d17-417d-b675-b35ecbd3d177 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:15.336288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da155ff-3d17-417d-b675-b35ecbd3d177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:15.336777 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da155ff-3d17-417d-b675-b35ecbd3d177 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:15.337638 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da155ff-3d17-417d-b675-b35ecbd3d177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:15.340262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da155ff-3d17-417d-b675-b35ecbd3d177 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:15.357416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1184/2350] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:31:15.361211 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46902dfe-b3ca-4770-8447-c8237b0ac988 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:15.374332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46902dfe-b3ca-4770-8447-c8237b0ac988 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:15.380769 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25111078-d71f-4440-9fd8-c43f7bbc2176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:15.381494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25111078-d71f-4440-9fd8-c43f7bbc2176 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:15.381807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25111078-d71f-4440-9fd8-c43f7bbc2176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:15.382591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25111078-d71f-4440-9fd8-c43f7bbc2176 admin admin] PATH_INFO: `/v3/projects/13f4b2186b7342c28a9b62c740750db6/users/796ccf5ac7fc429db3d181208c72f1c9/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:15.384685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25111078-d71f-4440-9fd8-c43f7bbc2176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f4b2186b7342c28a9b62c740750db6, user_id=796ccf5ac7fc429db3d181208c72f1c9, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:15.387504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25111078-d71f-4440-9fd8-c43f7bbc2176 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:15.420233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1167/2351] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:31:15 2022] PUT /identity/v3/projects/13f4b2186b7342c28a9b62c740750db6/users/796ccf5ac7fc429db3d181208c72f1c9/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:31:22.203342 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e6a38-4453-4d56-b392-667e1b83d6bb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:22.203889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e6a38-4453-4d56-b392-667e1b83d6bb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:22.204012 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e6a38-4453-4d56-b392-667e1b83d6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:22.239695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2a5e6a38-4453-4d56-b392-667e1b83d6bb None None] MFA Rules not processed for user `796ccf5ac7fc429db3d181208c72f1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:31:22.240955 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2a5e6a38-4453-4d56-b392-667e1b83d6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:22.358855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1185/2352] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:31:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:31:22.375178 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25111078-d71f-4440-9fd8-c43f7bbc2176 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:22.377057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-25111078-d71f-4440-9fd8-c43f7bbc2176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:22.470658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25111078-d71f-4440-9fd8-c43f7bbc2176 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:22.477233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76537a9c-205a-4c44-b63b-8b8453442196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:22.477957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76537a9c-205a-4c44-b63b-8b8453442196 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:22.478348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76537a9c-205a-4c44-b63b-8b8453442196 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:22.478699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76537a9c-205a-4c44-b63b-8b8453442196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:22.542365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76537a9c-205a-4c44-b63b-8b8453442196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:22.544013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76537a9c-205a-4c44-b63b-8b8453442196 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:22.554304 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1168/2353] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:31:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:31:23.588897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c77988-4d76-4f5f-9973-37c67bcd3c1d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:23.589364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c77988-4d76-4f5f-9973-37c67bcd3c1d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:23.589455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c77988-4d76-4f5f-9973-37c67bcd3c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:23.609322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-86c77988-4d76-4f5f-9973-37c67bcd3c1d None None] MFA Rules not processed for user `2e22d2226f77454fb3f155dbf3fdf471`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:31:23.610543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-86c77988-4d76-4f5f-9973-37c67bcd3c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:23.617594 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1186/2354] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:31:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:31:27.460056 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9b6df-7982-4941-ae00-25e0b53ed8b3 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:27.460571 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9b6df-7982-4941-ae00-25e0b53ed8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:27.460680 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9b6df-7982-4941-ae00-25e0b53ed8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:27.483345 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-1bf9b6df-7982-4941-ae00-25e0b53ed8b3 None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:31:27.484591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1bf9b6df-7982-4941-ae00-25e0b53ed8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:27.517690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:31:27.542228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1169/2355] 10.222.0.27 () {58 vars in 967 bytes} [Mon Sep 12 23:31:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:31:27.563926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c77988-4d76-4f5f-9973-37c67bcd3c1d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:27.565434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-86c77988-4d76-4f5f-9973-37c67bcd3c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:27.641107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c77988-4d76-4f5f-9973-37c67bcd3c1d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:27.649343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ed227-9eb1-4ac6-8820-de18eeb81050 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:27.650775 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed227-9eb1-4ac6-8820-de18eeb81050 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:27.651055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed227-9eb1-4ac6-8820-de18eeb81050 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:27.651338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed227-9eb1-4ac6-8820-de18eeb81050 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:27.660464 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ed227-9eb1-4ac6-8820-de18eeb81050 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:27.662053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ed227-9eb1-4ac6-8820-de18eeb81050 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:27.670761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1187/2356] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:31:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:31:32.636611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9b6df-7982-4941-ae00-25e0b53ed8b3 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:32.692640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9b6df-7982-4941-ae00-25e0b53ed8b3 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:32.697047 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987590f7-860a-437c-86ea-b5b2295646f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:32.697654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987590f7-860a-437c-86ea-b5b2295646f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:32.697969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987590f7-860a-437c-86ea-b5b2295646f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:32.698243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987590f7-860a-437c-86ea-b5b2295646f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:32.700013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-987590f7-860a-437c-86ea-b5b2295646f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:31:32.762584 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987590f7-860a-437c-86ea-b5b2295646f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:32.764025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987590f7-860a-437c-86ea-b5b2295646f1 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:32.775645 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1170/2357] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:31:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:31:37.666878 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:31:42.770147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=57 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:31:57.502220 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ed227-9eb1-4ac6-8820-de18eeb81050 service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:31:57.511850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ed227-9eb1-4ac6-8820-de18eeb81050 service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:31:57.515350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ee930-a3e0-42a4-b53e-c767a881495d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:31:57.515966 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ee930-a3e0-42a4-b53e-c767a881495d service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:31:57.516244 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ee930-a3e0-42a4-b53e-c767a881495d service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:31:57.516514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ee930-a3e0-42a4-b53e-c767a881495d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:31:57.525748 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ee930-a3e0-42a4-b53e-c767a881495d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:31:57.527155 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ee930-a3e0-42a4-b53e-c767a881495d service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:31:57.536723 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1188/2358] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:31:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:32:07.532480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:32:20.797807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db7c9f-22d1-405a-8890-cf24aa27dc0f None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:32:20.798324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db7c9f-22d1-405a-8890-cf24aa27dc0f None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:32:20.798428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db7c9f-22d1-405a-8890-cf24aa27dc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:32:20.817602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-16db7c9f-22d1-405a-8890-cf24aa27dc0f None None] MFA Rules not processed for user `2749a61b3dd04d89a5c2e4235aa96934`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:32:20.818825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-16db7c9f-22d1-405a-8890-cf24aa27dc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:32:20.873703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1171/2359] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:32:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:32:30.866952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=25 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:32:34.844488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad445b-d1de-4745-a594-9677cfdcb118 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:32:34.845353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad445b-d1de-4745-a594-9677cfdcb118 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:32:34.845806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad445b-d1de-4745-a594-9677cfdcb118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:32:34.876491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d0ad445b-d1de-4745-a594-9677cfdcb118 None None] MFA Rules not processed for user `0248519bf5b54ce3ba3b40f2c9a4c628`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:32:34.877738 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d0ad445b-d1de-4745-a594-9677cfdcb118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:32:34.949017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1189/2360] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:32:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:32:42.865961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b764b22-7006-49ce-a3c0-b9eee5d4d6dc None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:32:42.866456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b764b22-7006-49ce-a3c0-b9eee5d4d6dc None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:32:42.866559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b764b22-7006-49ce-a3c0-b9eee5d4d6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:32:42.885678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-6b764b22-7006-49ce-a3c0-b9eee5d4d6dc None None] MFA Rules not processed for user `7f56dde21c754cfe8253ec5c1b7b90d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:32:42.886868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6b764b22-7006-49ce-a3c0-b9eee5d4d6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:32:42.940539 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1172/2361] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:32:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:32:44.940902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=25 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:32:52.933862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=25 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:10.626456 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad445b-d1de-4745-a594-9677cfdcb118 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:10.628018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d0ad445b-d1de-4745-a594-9677cfdcb118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:10.644087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad445b-d1de-4745-a594-9677cfdcb118 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:10.649918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68b811-92b7-48b8-9212-e4f8de662568 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:10.650511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68b811-92b7-48b8-9212-e4f8de662568 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:10.650784 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68b811-92b7-48b8-9212-e4f8de662568 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:10.651309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68b811-92b7-48b8-9212-e4f8de662568 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:10.652969 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9b68b811-92b7-48b8-9212-e4f8de662568 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:10.663385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68b811-92b7-48b8-9212-e4f8de662568 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:10.664964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68b811-92b7-48b8-9212-e4f8de662568 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:10.673539 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1190/2362] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:13.343909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b764b22-7006-49ce-a3c0-b9eee5d4d6dc None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:13.345325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6b764b22-7006-49ce-a3c0-b9eee5d4d6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:13.401599 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b764b22-7006-49ce-a3c0-b9eee5d4d6dc None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:13.405203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5e523-8ae7-48cf-ab31-a2b120f7ee0c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61407406ef242a78030f79532fe2247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:13.405802 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5e523-8ae7-48cf-ab31-a2b120f7ee0c service glance] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:13.406083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5e523-8ae7-48cf-ab31-a2b120f7ee0c service glance] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:13.406358 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5e523-8ae7-48cf-ab31-a2b120f7ee0c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:13.408125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-20e5e523-8ae7-48cf-ab31-a2b120f7ee0c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:13.422113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5e523-8ae7-48cf-ab31-a2b120f7ee0c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:13.423588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5e523-8ae7-48cf-ab31-a2b120f7ee0c service glance] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:13.434752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1173/2363] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:13.538520 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68b811-92b7-48b8-9212-e4f8de662568 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:13.539963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9b68b811-92b7-48b8-9212-e4f8de662568 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:13.597845 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68b811-92b7-48b8-9212-e4f8de662568 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:13.602117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e5397-40f1-479f-89a3-f10e7bacfa42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:13.603130 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e5397-40f1-479f-89a3-f10e7bacfa42 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:13.603766 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e5397-40f1-479f-89a3-f10e7bacfa42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:13.604265 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e5397-40f1-479f-89a3-f10e7bacfa42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:13.606840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-198e5397-40f1-479f-89a3-f10e7bacfa42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:13.623334 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e5397-40f1-479f-89a3-f10e7bacfa42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:13.624954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e5397-40f1-479f-89a3-f10e7bacfa42 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:13.637025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1191/2364] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:17.403965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5e523-8ae7-48cf-ab31-a2b120f7ee0c service glance] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:17.416038 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5e523-8ae7-48cf-ab31-a2b120f7ee0c service glance] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:17.421506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e80c0b-2375-4f82-be5c-6d4a9dfb5019 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:17.422095 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e80c0b-2375-4f82-be5c-6d4a9dfb5019 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:17.422376 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e80c0b-2375-4f82-be5c-6d4a9dfb5019 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:17.422650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e80c0b-2375-4f82-be5c-6d4a9dfb5019 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:17.424398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-56e80c0b-2375-4f82-be5c-6d4a9dfb5019 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:17.435273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e80c0b-2375-4f82-be5c-6d4a9dfb5019 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:17.436902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e80c0b-2375-4f82-be5c-6d4a9dfb5019 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:17.445663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1174/2365] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:23.631495 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=32 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:27.441575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=32 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:32.274789 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e5397-40f1-479f-89a3-f10e7bacfa42 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:32.287412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e5397-40f1-479f-89a3-f10e7bacfa42 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:32.293612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b1fee-94f8-4353-bfe6-4416d83d231c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:32.294390 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b1fee-94f8-4353-bfe6-4416d83d231c service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:32.294675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b1fee-94f8-4353-bfe6-4416d83d231c service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:32.294950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b1fee-94f8-4353-bfe6-4416d83d231c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:32.297110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c16b1fee-94f8-4353-bfe6-4416d83d231c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:32.308981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b1fee-94f8-4353-bfe6-4416d83d231c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:32.310564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b1fee-94f8-4353-bfe6-4416d83d231c service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:32.319507 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1192/2366] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:32.704873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e80c0b-2375-4f82-be5c-6d4a9dfb5019 service nova] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:32.706356 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-56e80c0b-2375-4f82-be5c-6d4a9dfb5019 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:32.718078 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e80c0b-2375-4f82-be5c-6d4a9dfb5019 service nova] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:32.721592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc294678-ee87-4c45-a74d-1a035082e7dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:32.722173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc294678-ee87-4c45-a74d-1a035082e7dd service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:32.722450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc294678-ee87-4c45-a74d-1a035082e7dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:32.722721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc294678-ee87-4c45-a74d-1a035082e7dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:32.724358 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bc294678-ee87-4c45-a74d-1a035082e7dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:32.787100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc294678-ee87-4c45-a74d-1a035082e7dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:32.789231 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc294678-ee87-4c45-a74d-1a035082e7dd service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:32.798539 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1175/2367] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:36.219023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b1fee-94f8-4353-bfe6-4416d83d231c service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:36.284331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc294678-ee87-4c45-a74d-1a035082e7dd service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:36.286773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bc294678-ee87-4c45-a74d-1a035082e7dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:36.306817 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b1fee-94f8-4353-bfe6-4416d83d231c service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:36.309283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc294678-ee87-4c45-a74d-1a035082e7dd service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:36.312561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b35402-6a03-4005-a5ab-48d4ff1b67ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b3b0a0317b14e29a32eb6ba6b89c2aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:36.313231 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b35402-6a03-4005-a5ab-48d4ff1b67ba service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:36.313524 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b35402-6a03-4005-a5ab-48d4ff1b67ba service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:36.313805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b35402-6a03-4005-a5ab-48d4ff1b67ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:36.314922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a77fc-8e52-4afd-8541-2b610f1ecf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:36.315556 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-45b35402-6a03-4005-a5ab-48d4ff1b67ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:36.315713 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a77fc-8e52-4afd-8541-2b610f1ecf6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:36.315974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a77fc-8e52-4afd-8541-2b610f1ecf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:36.316619 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a77fc-8e52-4afd-8541-2b610f1ecf6b admin admin] PATH_INFO: `/v3/users/0248519bf5b54ce3ba3b40f2c9a4c628` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:36.318075 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a77fc-8e52-4afd-8541-2b610f1ecf6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0248519bf5b54ce3ba3b40f2c9a4c628)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:36.320874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a77fc-8e52-4afd-8541-2b610f1ecf6b admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:36.330278 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b35402-6a03-4005-a5ab-48d4ff1b67ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:36.331960 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b35402-6a03-4005-a5ab-48d4ff1b67ba service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:36.346006 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1193/2368] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:36.398245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-cb9a77fc-8e52-4afd-8541-2b610f1ecf6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0248519bf5b54ce3ba3b40f2c9a4c628'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:36.418472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-cb9a77fc-8e52-4afd-8541-2b610f1ecf6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0248519bf5b54ce3ba3b40f2c9a4c628'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:36.426767 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-cb9a77fc-8e52-4afd-8541-2b610f1ecf6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0248519bf5b54ce3ba3b40f2c9a4c628'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:36.427781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-cb9a77fc-8e52-4afd-8541-2b610f1ecf6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0248519bf5b54ce3ba3b40f2c9a4c628'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:36.442982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-cb9a77fc-8e52-4afd-8541-2b610f1ecf6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0248519bf5b54ce3ba3b40f2c9a4c628'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:36.450147 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1176/2369] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:33:36 2022] DELETE /identity/v3/users/0248519bf5b54ce3ba3b40f2c9a4c628 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:36.622363 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e9dd6-d51b-4835-82b1-d7c3a949d011 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:36.623006 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e9dd6-d51b-4835-82b1-d7c3a949d011 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:36.623323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e9dd6-d51b-4835-82b1-d7c3a949d011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:36.645638 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-014e9dd6-d51b-4835-82b1-d7c3a949d011 None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:36.646964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-014e9dd6-d51b-4835-82b1-d7c3a949d011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:36.652842 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a77fc-8e52-4afd-8541-2b610f1ecf6b admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:36.654476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-cb9a77fc-8e52-4afd-8541-2b610f1ecf6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:36.749800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a77fc-8e52-4afd-8541-2b610f1ecf6b admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:36.754772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56595cb9-37b1-4a7e-b7c7-306908c42a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:36.755418 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56595cb9-37b1-4a7e-b7c7-306908c42a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:36.755735 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56595cb9-37b1-4a7e-b7c7-306908c42a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:36.756014 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56595cb9-37b1-4a7e-b7c7-306908c42a2d admin admin] PATH_INFO: `/v3/projects/adc049ff85c14935a9f68be76cd61c05` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:36.757846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56595cb9-37b1-4a7e-b7c7-306908c42a2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=adc049ff85c14935a9f68be76cd61c05)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:36.759794 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56595cb9-37b1-4a7e-b7c7-306908c42a2d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:36.769306 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:36.782287 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1194/2370] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:33:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:36.824384 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-56595cb9-37b1-4a7e-b7c7-306908c42a2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adc049ff85c14935a9f68be76cd61c05', 'request_id': 'req-56595cb9-37b1-4a7e-b7c7-306908c42a2d'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:36.833730 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-56595cb9-37b1-4a7e-b7c7-306908c42a2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adc049ff85c14935a9f68be76cd61c05', 'request_id': 'req-56595cb9-37b1-4a7e-b7c7-306908c42a2d'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:36.845840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-56595cb9-37b1-4a7e-b7c7-306908c42a2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adc049ff85c14935a9f68be76cd61c05', 'request_id': 'req-56595cb9-37b1-4a7e-b7c7-306908c42a2d'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:36.853616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e9dd6-d51b-4835-82b1-d7c3a949d011 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:36.855272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-014e9dd6-d51b-4835-82b1-d7c3a949d011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:36.860725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-56595cb9-37b1-4a7e-b7c7-306908c42a2d admin admin] The token cache is being invalidate because project adc049ff85c14935a9f68be76cd61c05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:33:36.861156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-56595cb9-37b1-4a7e-b7c7-306908c42a2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:36.861734 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-56595cb9-37b1-4a7e-b7c7-306908c42a2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:36.867010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1177/2371] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:33:36 2022] DELETE /identity/v3/projects/adc049ff85c14935a9f68be76cd61c05 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:36.868842 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce08ca-b3ce-456e-9045-fae4903eb52d None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:36.869141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce08ca-b3ce-456e-9045-fae4903eb52d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:36.869424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce08ca-b3ce-456e-9045-fae4903eb52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:36.894790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-58ce08ca-b3ce-456e-9045-fae4903eb52d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:36.896060 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-58ce08ca-b3ce-456e-9045-fae4903eb52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:36.955011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e9dd6-d51b-4835-82b1-d7c3a949d011 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:36.977819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:36.979549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1178/2372] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:36.987597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e18aa1-7e4f-44a5-8940-a2233e6eb9ca None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:36.988098 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e18aa1-7e4f-44a5-8940-a2233e6eb9ca None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:36.988558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e18aa1-7e4f-44a5-8940-a2233e6eb9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:37.011157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-d1e18aa1-7e4f-44a5-8940-a2233e6eb9ca None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:37.012422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d1e18aa1-7e4f-44a5-8940-a2233e6eb9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:37.020723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1179/2373] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:33:37.025233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e18aa1-7e4f-44a5-8940-a2233e6eb9ca None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:37.027249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafac711-3312-45d8-b966-e43993cde227 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:37.028163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafac711-3312-45d8-b966-e43993cde227 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:37.028675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafac711-3312-45d8-b966-e43993cde227 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:37.029175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafac711-3312-45d8-b966-e43993cde227 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:37.032018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eafac711-3312-45d8-b966-e43993cde227 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:37.034781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e18aa1-7e4f-44a5-8940-a2233e6eb9ca None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:37.043874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e042371-255d-4735-8526-03fdf0f493c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:37.044582 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e042371-255d-4735-8526-03fdf0f493c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:37.044767 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e042371-255d-4735-8526-03fdf0f493c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:37.045037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e042371-255d-4735-8526-03fdf0f493c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:37.045800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e042371-255d-4735-8526-03fdf0f493c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:33:37.046132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e042371-255d-4735-8526-03fdf0f493c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:37.047755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e042371-255d-4735-8526-03fdf0f493c7 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:37.059592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1180/2374] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:33:37.063886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e042371-255d-4735-8526-03fdf0f493c7 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:37.073573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e042371-255d-4735-8526-03fdf0f493c7 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:37.078741 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86096fd-5e5f-4db6-a670-e3a76ea9cb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:37.079321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86096fd-5e5f-4db6-a670-e3a76ea9cb9b admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:37.079767 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86096fd-5e5f-4db6-a670-e3a76ea9cb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:37.080044 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86096fd-5e5f-4db6-a670-e3a76ea9cb9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:37.080836 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86096fd-5e5f-4db6-a670-e3a76ea9cb9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:37.082744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86096fd-5e5f-4db6-a670-e3a76ea9cb9b admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:37.094466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafac711-3312-45d8-b966-e43993cde227 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:37.096094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafac711-3312-45d8-b966-e43993cde227 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:37.105528 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1195/2375] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:33:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:33:37.115603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1181/2376] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:33:37 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:33:37.117399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafac711-3312-45d8-b966-e43993cde227 service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:37.132640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafac711-3312-45d8-b966-e43993cde227 service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:37.215196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf569be-e019-4e2a-8586-6ba2bdbf6561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:37.215757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf569be-e019-4e2a-8586-6ba2bdbf6561 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:37.216045 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf569be-e019-4e2a-8586-6ba2bdbf6561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:37.216324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf569be-e019-4e2a-8586-6ba2bdbf6561 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:37.216958 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf569be-e019-4e2a-8586-6ba2bdbf6561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:37.218878 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf569be-e019-4e2a-8586-6ba2bdbf6561 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:37.262858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1196/2377] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:33:37 2022] POST /identity/v3/users => generated 283 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:33:37.264590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86096fd-5e5f-4db6-a670-e3a76ea9cb9b admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:37.280993 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86096fd-5e5f-4db6-a670-e3a76ea9cb9b admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:37.286746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2101b-190f-43f4-9af5-285d5f63de3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:37.288913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2101b-190f-43f4-9af5-285d5f63de3a admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:37.289551 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2101b-190f-43f4-9af5-285d5f63de3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:37.290038 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2101b-190f-43f4-9af5-285d5f63de3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:37.291115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2101b-190f-43f4-9af5-285d5f63de3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:37.292959 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2101b-190f-43f4-9af5-285d5f63de3a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:37.306133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1182/2378] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:33:37.310127 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf569be-e019-4e2a-8586-6ba2bdbf6561 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:37.324186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf569be-e019-4e2a-8586-6ba2bdbf6561 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:37.329243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6349d-ae4e-4ef2-a7db-f03af1cfed0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:37.329854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6349d-ae4e-4ef2-a7db-f03af1cfed0d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:37.330327 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6349d-ae4e-4ef2-a7db-f03af1cfed0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:37.330453 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6349d-ae4e-4ef2-a7db-f03af1cfed0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:37.331049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6349d-ae4e-4ef2-a7db-f03af1cfed0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:37.332696 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6349d-ae4e-4ef2-a7db-f03af1cfed0d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:37.345026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1197/2379] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:33:37.349199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2101b-190f-43f4-9af5-285d5f63de3a admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:37.360481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2101b-190f-43f4-9af5-285d5f63de3a admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:37.365455 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd306805-fef0-4de5-9b05-655fe1323cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:37.366384 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd306805-fef0-4de5-9b05-655fe1323cce admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:37.366823 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd306805-fef0-4de5-9b05-655fe1323cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:37.367104 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd306805-fef0-4de5-9b05-655fe1323cce admin admin] PATH_INFO: `/v3/projects/3dbc77a092e04fb481eaf85233af1773/users/a817ac3985fb4be085b5a66e5afaa1b0/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:37.386440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd306805-fef0-4de5-9b05-655fe1323cce admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dbc77a092e04fb481eaf85233af1773, user_id=a817ac3985fb4be085b5a66e5afaa1b0, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:37.389108 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd306805-fef0-4de5-9b05-655fe1323cce admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:37.404370 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1183/2380] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:33:37 2022] PUT /identity/v3/projects/3dbc77a092e04fb481eaf85233af1773/users/a817ac3985fb4be085b5a66e5afaa1b0/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:37.415605 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be817a-db62-4209-a755-91fb9c3941ce None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:37.415915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be817a-db62-4209-a755-91fb9c3941ce None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:37.416195 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be817a-db62-4209-a755-91fb9c3941ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:37.449072 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c1be817a-db62-4209-a755-91fb9c3941ce None None] MFA Rules not processed for user `a817ac3985fb4be085b5a66e5afaa1b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:37.450338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c1be817a-db62-4209-a755-91fb9c3941ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:37.492822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:37.543884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1198/2381] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:37.544807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd306805-fef0-4de5-9b05-655fe1323cce admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:37.574801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:37.621690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd306805-fef0-4de5-9b05-655fe1323cce admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:37.627148 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ad3ab-1f00-485c-a790-7ae6198450ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:37.627877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ad3ab-1f00-485c-a790-7ae6198450ac admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:37.628228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ad3ab-1f00-485c-a790-7ae6198450ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:37.628576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ad3ab-1f00-485c-a790-7ae6198450ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:37.629310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ad3ab-1f00-485c-a790-7ae6198450ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:37.631322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ad3ab-1f00-485c-a790-7ae6198450ac admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:37.664693 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1184/2382] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:33:37 2022] POST /identity/v3/projects => generated 363 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:33:37.668761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be817a-db62-4209-a755-91fb9c3941ce None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:37.684301 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be817a-db62-4209-a755-91fb9c3941ce None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:37.756533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7b85e-eb0e-49a3-9eee-4c9eb69df23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:37.757917 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7b85e-eb0e-49a3-9eee-4c9eb69df23e admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:37.758314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7b85e-eb0e-49a3-9eee-4c9eb69df23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:37.758685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7b85e-eb0e-49a3-9eee-4c9eb69df23e admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:37.759662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7b85e-eb0e-49a3-9eee-4c9eb69df23e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:37.761720 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7b85e-eb0e-49a3-9eee-4c9eb69df23e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:37.789766 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1199/2383] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:33:37 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:33:37.791168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ad3ab-1f00-485c-a790-7ae6198450ac admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:37.801982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ad3ab-1f00-485c-a790-7ae6198450ac admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:37.807697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a0e59-aaea-465e-8ae7-31d0af6ca13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:37.808461 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a0e59-aaea-465e-8ae7-31d0af6ca13d admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:37.808754 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a0e59-aaea-465e-8ae7-31d0af6ca13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:37.809030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a0e59-aaea-465e-8ae7-31d0af6ca13d admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:37.809686 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a0e59-aaea-465e-8ae7-31d0af6ca13d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:37.811431 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a0e59-aaea-465e-8ae7-31d0af6ca13d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:37.827203 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1185/2384] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:33:37.828584 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7b85e-eb0e-49a3-9eee-4c9eb69df23e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:37.840567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7b85e-eb0e-49a3-9eee-4c9eb69df23e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:37.846250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a000a-8486-48a0-843f-201fbce0b467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:37.847122 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a000a-8486-48a0-843f-201fbce0b467 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:37.847795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a000a-8486-48a0-843f-201fbce0b467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:37.848228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a000a-8486-48a0-843f-201fbce0b467 admin admin] PATH_INFO: `/v3/projects/5537c1a7bb08471c89bcfc2df518a695/users/65e3aca7643040d7b5b963c6d2232c0c/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:37.868211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a000a-8486-48a0-843f-201fbce0b467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5537c1a7bb08471c89bcfc2df518a695, user_id=65e3aca7643040d7b5b963c6d2232c0c, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:37.871252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a000a-8486-48a0-843f-201fbce0b467 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:37.889776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1200/2385] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:33:37 2022] PUT /identity/v3/projects/5537c1a7bb08471c89bcfc2df518a695/users/65e3aca7643040d7b5b963c6d2232c0c/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:37.893508 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a0e59-aaea-465e-8ae7-31d0af6ca13d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:37.906851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a0e59-aaea-465e-8ae7-31d0af6ca13d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:37.983460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37773f-afc1-49a1-8e44-25c3b4aae017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:37.984103 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37773f-afc1-49a1-8e44-25c3b4aae017 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:37.984404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37773f-afc1-49a1-8e44-25c3b4aae017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:37.984675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37773f-afc1-49a1-8e44-25c3b4aae017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:37.985315 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37773f-afc1-49a1-8e44-25c3b4aae017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:37.986948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37773f-afc1-49a1-8e44-25c3b4aae017 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:37.997999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1186/2386] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:33:38.001962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a000a-8486-48a0-843f-201fbce0b467 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:38.013886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a000a-8486-48a0-843f-201fbce0b467 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:38.019662 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8042bb2-4977-4d96-91b1-1f0111588441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:38.019948 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8042bb2-4977-4d96-91b1-1f0111588441 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:38.020235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8042bb2-4977-4d96-91b1-1f0111588441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:38.020511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8042bb2-4977-4d96-91b1-1f0111588441 admin admin] PATH_INFO: `/v3/projects/5537c1a7bb08471c89bcfc2df518a695/users/65e3aca7643040d7b5b963c6d2232c0c/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:38.022465 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8042bb2-4977-4d96-91b1-1f0111588441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5537c1a7bb08471c89bcfc2df518a695, user_id=65e3aca7643040d7b5b963c6d2232c0c, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:38.026180 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8042bb2-4977-4d96-91b1-1f0111588441 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:38.047923 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1201/2387] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:33:38 2022] PUT /identity/v3/projects/5537c1a7bb08471c89bcfc2df518a695/users/65e3aca7643040d7b5b963c6d2232c0c/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:38.049485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c3bf-238f-4e67-869b-7e94fa3ee70e None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:38.049804 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c3bf-238f-4e67-869b-7e94fa3ee70e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:38.050064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c3bf-238f-4e67-869b-7e94fa3ee70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:38.087069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-79e8c3bf-238f-4e67-869b-7e94fa3ee70e None None] MFA Rules not processed for user `65e3aca7643040d7b5b963c6d2232c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:38.088317 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-79e8c3bf-238f-4e67-869b-7e94fa3ee70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:38.168356 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1187/2388] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:38.179813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8042bb2-4977-4d96-91b1-1f0111588441 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:38.181889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d8042bb2-4977-4d96-91b1-1f0111588441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:38.253077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8042bb2-4977-4d96-91b1-1f0111588441 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:38.257361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2244d2-6e00-4c62-9745-01e6ec3cb2a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:38.257976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2244d2-6e00-4c62-9745-01e6ec3cb2a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:38.258251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2244d2-6e00-4c62-9745-01e6ec3cb2a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:38.258520 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2244d2-6e00-4c62-9745-01e6ec3cb2a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:38.269343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2244d2-6e00-4c62-9745-01e6ec3cb2a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:38.270713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2244d2-6e00-4c62-9745-01e6ec3cb2a6 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:38.284245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1202/2389] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:38.810793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8c3bf-238f-4e67-869b-7e94fa3ee70e None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:38.836874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:38.921758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8c3bf-238f-4e67-869b-7e94fa3ee70e None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:38.934255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6c54d-2a16-4101-b7d5-6ddc38608207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:38.934914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6c54d-2a16-4101-b7d5-6ddc38608207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:38.935192 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6c54d-2a16-4101-b7d5-6ddc38608207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:38.935710 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6c54d-2a16-4101-b7d5-6ddc38608207 admin admin] PATH_INFO: `/v3/users/a817ac3985fb4be085b5a66e5afaa1b0` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:38.936952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d6c54d-2a16-4101-b7d5-6ddc38608207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a817ac3985fb4be085b5a66e5afaa1b0)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:38.942647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d6c54d-2a16-4101-b7d5-6ddc38608207 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:39.023862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-66d6c54d-2a16-4101-b7d5-6ddc38608207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a817ac3985fb4be085b5a66e5afaa1b0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:39.037462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-66d6c54d-2a16-4101-b7d5-6ddc38608207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a817ac3985fb4be085b5a66e5afaa1b0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:39.045476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-66d6c54d-2a16-4101-b7d5-6ddc38608207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a817ac3985fb4be085b5a66e5afaa1b0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:39.046412 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-66d6c54d-2a16-4101-b7d5-6ddc38608207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a817ac3985fb4be085b5a66e5afaa1b0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:39.054191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-66d6c54d-2a16-4101-b7d5-6ddc38608207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a817ac3985fb4be085b5a66e5afaa1b0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:39.061685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1188/2390] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:33:38 2022] DELETE /identity/v3/users/a817ac3985fb4be085b5a66e5afaa1b0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:39.066904 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc0e4-9eb2-43cc-8923-b432343bd36b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:39.067417 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc0e4-9eb2-43cc-8923-b432343bd36b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:39.067997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc0e4-9eb2-43cc-8923-b432343bd36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:39.094820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-104fc0e4-9eb2-43cc-8923-b432343bd36b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:39.096281 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-104fc0e4-9eb2-43cc-8923-b432343bd36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:39.130879 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:39.201379 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1203/2391] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:39.203869 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6c54d-2a16-4101-b7d5-6ddc38608207 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:39.205509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-66d6c54d-2a16-4101-b7d5-6ddc38608207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:39.278928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6c54d-2a16-4101-b7d5-6ddc38608207 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:39.283554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488e38a-7d90-4b53-b921-9d5bc7478c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:39.284322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e38a-7d90-4b53-b921-9d5bc7478c4e service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:39.284704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e38a-7d90-4b53-b921-9d5bc7478c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:39.285079 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e38a-7d90-4b53-b921-9d5bc7478c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:39.297241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488e38a-7d90-4b53-b921-9d5bc7478c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:39.299804 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488e38a-7d90-4b53-b921-9d5bc7478c4e service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:39.318588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1189/2392] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:39.657146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fc0e4-9eb2-43cc-8923-b432343bd36b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:39.659105 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-104fc0e4-9eb2-43cc-8923-b432343bd36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:39.674755 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fc0e4-9eb2-43cc-8923-b432343bd36b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:39.681790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:39.682910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:39.683378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:39.683941 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] PATH_INFO: `/v3/projects/3dbc77a092e04fb481eaf85233af1773` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:39.686404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dbc77a092e04fb481eaf85233af1773)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:39.688769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:39.753074 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dbc77a092e04fb481eaf85233af1773', 'request_id': 'req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:39.763548 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dbc77a092e04fb481eaf85233af1773', 'request_id': 'req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:39.772248 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dbc77a092e04fb481eaf85233af1773', 'request_id': 'req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:39.787030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] The token cache is being invalidate because project 3dbc77a092e04fb481eaf85233af1773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:33:39.787328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:39.788023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:39.789522 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1204/2393] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:33:39 2022] DELETE /identity/v3/projects/3dbc77a092e04fb481eaf85233af1773 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:39.793557 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488e38a-7d90-4b53-b921-9d5bc7478c4e service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:39.805337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488e38a-7d90-4b53-b921-9d5bc7478c4e service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:39.852551 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:39.883825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de78628-eed8-432b-b56a-aa9ca45e81a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:39.884662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de78628-eed8-432b-b56a-aa9ca45e81a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:39.885023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de78628-eed8-432b-b56a-aa9ca45e81a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:39.885426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de78628-eed8-432b-b56a-aa9ca45e81a2 admin admin] PATH_INFO: `/v3/users/65e3aca7643040d7b5b963c6d2232c0c` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:39.886692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de78628-eed8-432b-b56a-aa9ca45e81a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65e3aca7643040d7b5b963c6d2232c0c)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:39.888755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de78628-eed8-432b-b56a-aa9ca45e81a2 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:39.961756 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4de78628-eed8-432b-b56a-aa9ca45e81a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65e3aca7643040d7b5b963c6d2232c0c'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:39.995049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4de78628-eed8-432b-b56a-aa9ca45e81a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65e3aca7643040d7b5b963c6d2232c0c'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:40.021826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4de78628-eed8-432b-b56a-aa9ca45e81a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65e3aca7643040d7b5b963c6d2232c0c'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:40.022489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4de78628-eed8-432b-b56a-aa9ca45e81a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65e3aca7643040d7b5b963c6d2232c0c'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:40.031288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4de78628-eed8-432b-b56a-aa9ca45e81a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65e3aca7643040d7b5b963c6d2232c0c'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:40.037969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1190/2394] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:33:39 2022] DELETE /identity/v3/users/65e3aca7643040d7b5b963c6d2232c0c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:40.393430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:40.396027 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:40.440968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0ab98-79d3-48f3-9c7f-f1ba0c13d8d2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:40.546288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4cf0a-eca2-4fb4-9df3-725ac233a905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:40.546923 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4cf0a-eca2-4fb4-9df3-725ac233a905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:40.547203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4cf0a-eca2-4fb4-9df3-725ac233a905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:40.547470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4cf0a-eca2-4fb4-9df3-725ac233a905 admin admin] PATH_INFO: `/v3/projects/5537c1a7bb08471c89bcfc2df518a695` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:40.548677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4cf0a-eca2-4fb4-9df3-725ac233a905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5537c1a7bb08471c89bcfc2df518a695)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:40.550632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4cf0a-eca2-4fb4-9df3-725ac233a905 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:40.632333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-50b4cf0a-eca2-4fb4-9df3-725ac233a905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5537c1a7bb08471c89bcfc2df518a695', 'request_id': 'req-50b4cf0a-eca2-4fb4-9df3-725ac233a905'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:40.637914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:40.641799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-50b4cf0a-eca2-4fb4-9df3-725ac233a905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5537c1a7bb08471c89bcfc2df518a695', 'request_id': 'req-50b4cf0a-eca2-4fb4-9df3-725ac233a905'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:40.659122 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-50b4cf0a-eca2-4fb4-9df3-725ac233a905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5537c1a7bb08471c89bcfc2df518a695', 'request_id': 'req-50b4cf0a-eca2-4fb4-9df3-725ac233a905'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:40.686219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-50b4cf0a-eca2-4fb4-9df3-725ac233a905 admin admin] The token cache is being invalidate because project 5537c1a7bb08471c89bcfc2df518a695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:33:40.686697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-50b4cf0a-eca2-4fb4-9df3-725ac233a905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:40.687625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-50b4cf0a-eca2-4fb4-9df3-725ac233a905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:40.689596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1205/2395] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:33:40 2022] DELETE /identity/v3/projects/5537c1a7bb08471c89bcfc2df518a695 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:40.694186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b9cc7-faa3-4fe8-a33e-8f208b28f8f7 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:40.695127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b9cc7-faa3-4fe8-a33e-8f208b28f8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:40.695407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b9cc7-faa3-4fe8-a33e-8f208b28f8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:40.720032 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-053b9cc7-faa3-4fe8-a33e-8f208b28f8f7 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:40.721594 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-053b9cc7-faa3-4fe8-a33e-8f208b28f8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:40.847626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1191/2396] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:40.852594 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bfe8a-a813-496e-8b94-65219bdd824e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:40.852888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bfe8a-a813-496e-8b94-65219bdd824e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:40.853221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bfe8a-a813-496e-8b94-65219bdd824e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:40.895361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-618bfe8a-a813-496e-8b94-65219bdd824e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:40.896859 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-618bfe8a-a813-496e-8b94-65219bdd824e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:40.907083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1206/2397] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:40.911733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b9cc7-faa3-4fe8-a33e-8f208b28f8f7 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:40.927086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b9cc7-faa3-4fe8-a33e-8f208b28f8f7 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:40.955292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934ab0-5461-4780-891b-5ca199c1425b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:40.956099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934ab0-5461-4780-891b-5ca199c1425b admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:40.956387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934ab0-5461-4780-891b-5ca199c1425b admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:40.956663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934ab0-5461-4780-891b-5ca199c1425b admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:40.957322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59934ab0-5461-4780-891b-5ca199c1425b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:33:40.957631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59934ab0-5461-4780-891b-5ca199c1425b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:40.959379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59934ab0-5461-4780-891b-5ca199c1425b admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:40.987155 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1192/2398] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:33:40.992844 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bfe8a-a813-496e-8b94-65219bdd824e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:41.006531 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bfe8a-a813-496e-8b94-65219bdd824e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:41.016135 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886672e-04c6-453a-8886-1f8098423c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:41.017451 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886672e-04c6-453a-8886-1f8098423c74 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:41.018602 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886672e-04c6-453a-8886-1f8098423c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:41.018954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886672e-04c6-453a-8886-1f8098423c74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:41.020141 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886672e-04c6-453a-8886-1f8098423c74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:41.023208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886672e-04c6-453a-8886-1f8098423c74 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:41.073166 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1207/2399] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:33:40 2022] POST /identity/v3/projects => generated 365 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:33:41.077566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934ab0-5461-4780-891b-5ca199c1425b admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:41.089103 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934ab0-5461-4780-891b-5ca199c1425b admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:41.160622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:41.185141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d415b0f-cfa5-45be-8c37-4cf9f740ada9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:41.185760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d415b0f-cfa5-45be-8c37-4cf9f740ada9 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:41.186089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d415b0f-cfa5-45be-8c37-4cf9f740ada9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:41.186375 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d415b0f-cfa5-45be-8c37-4cf9f740ada9 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:41.187200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d415b0f-cfa5-45be-8c37-4cf9f740ada9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:41.189154 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d415b0f-cfa5-45be-8c37-4cf9f740ada9 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:41.226734 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1193/2400] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:33:41 2022] POST /identity/v3/users => generated 283 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:33:41.230783 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886672e-04c6-453a-8886-1f8098423c74 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:41.241907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886672e-04c6-453a-8886-1f8098423c74 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:41.250554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d0fcd-8a42-4b53-a852-3e6f1213f9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:41.251197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d0fcd-8a42-4b53-a852-3e6f1213f9ba admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:41.251798 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d0fcd-8a42-4b53-a852-3e6f1213f9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:41.252079 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d0fcd-8a42-4b53-a852-3e6f1213f9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:41.252744 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d0fcd-8a42-4b53-a852-3e6f1213f9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:41.257480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d0fcd-8a42-4b53-a852-3e6f1213f9ba admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:41.276500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1208/2401] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:33:41.280268 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d415b0f-cfa5-45be-8c37-4cf9f740ada9 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:41.297213 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d415b0f-cfa5-45be-8c37-4cf9f740ada9 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:41.303500 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677b711-48d9-4ee4-ba84-172266163392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:41.304218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677b711-48d9-4ee4-ba84-172266163392 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:41.304511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677b711-48d9-4ee4-ba84-172266163392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:41.304846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677b711-48d9-4ee4-ba84-172266163392 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:41.305443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0677b711-48d9-4ee4-ba84-172266163392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:41.307087 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0677b711-48d9-4ee4-ba84-172266163392 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:41.322718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1194/2402] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:33:41.323974 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d0fcd-8a42-4b53-a852-3e6f1213f9ba admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:41.336749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d0fcd-8a42-4b53-a852-3e6f1213f9ba admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:41.342235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff14335-e9a7-47b2-a687-f6078e4928db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:41.343033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff14335-e9a7-47b2-a687-f6078e4928db admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:41.344643 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff14335-e9a7-47b2-a687-f6078e4928db admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:41.344924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff14335-e9a7-47b2-a687-f6078e4928db admin admin] PATH_INFO: `/v3/projects/f76f07f7718148c79260627f14f32b81/users/4f74a7bc59f8444887a39720236e0adc/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:41.362764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff14335-e9a7-47b2-a687-f6078e4928db admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76f07f7718148c79260627f14f32b81, user_id=4f74a7bc59f8444887a39720236e0adc, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:41.366815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff14335-e9a7-47b2-a687-f6078e4928db admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:41.387607 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1209/2403] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:33:41 2022] PUT /identity/v3/projects/f76f07f7718148c79260627f14f32b81/users/4f74a7bc59f8444887a39720236e0adc/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:41.396339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b7b1e-d60c-4edf-84f1-cecd34399eba None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:41.396981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b7b1e-d60c-4edf-84f1-cecd34399eba None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:41.397460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b7b1e-d60c-4edf-84f1-cecd34399eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:41.427645 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-a47b7b1e-d60c-4edf-84f1-cecd34399eba None None] MFA Rules not processed for user `4f74a7bc59f8444887a39720236e0adc`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:41.429636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a47b7b1e-d60c-4edf-84f1-cecd34399eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:41.504373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1195/2404] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:41.506301 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff14335-e9a7-47b2-a687-f6078e4928db admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:41.576804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff14335-e9a7-47b2-a687-f6078e4928db admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:41.584416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7588e-4e98-4511-977b-4c9def18182d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:41.585555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7588e-4e98-4511-977b-4c9def18182d admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:41.585976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7588e-4e98-4511-977b-4c9def18182d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:41.586228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7588e-4e98-4511-977b-4c9def18182d admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:41.586989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7588e-4e98-4511-977b-4c9def18182d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:41.589292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7588e-4e98-4511-977b-4c9def18182d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:41.625940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1210/2405] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:33:41 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:33:41.630678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b7b1e-d60c-4edf-84f1-cecd34399eba None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:41.643116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b7b1e-d60c-4edf-84f1-cecd34399eba None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:41.731855 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd614d8d-ff6f-40c3-8ffb-e858ce38d1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:41.732686 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614d8d-ff6f-40c3-8ffb-e858ce38d1f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:41.733166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614d8d-ff6f-40c3-8ffb-e858ce38d1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:41.733528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614d8d-ff6f-40c3-8ffb-e858ce38d1f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:41.734316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd614d8d-ff6f-40c3-8ffb-e858ce38d1f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:41.736815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd614d8d-ff6f-40c3-8ffb-e858ce38d1f9 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:41.768350 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1196/2406] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:33:41 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:33:41.769158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7588e-4e98-4511-977b-4c9def18182d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:41.786761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7588e-4e98-4511-977b-4c9def18182d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:41.793002 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdedd8-5322-4608-ab0d-3e367c2b4976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:41.793919 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdedd8-5322-4608-ab0d-3e367c2b4976 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:41.794423 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdedd8-5322-4608-ab0d-3e367c2b4976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:41.794917 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abdedd8-5322-4608-ab0d-3e367c2b4976 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:41.795987 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abdedd8-5322-4608-ab0d-3e367c2b4976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:41.798000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abdedd8-5322-4608-ab0d-3e367c2b4976 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:41.816900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1211/2407] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:33:41.817116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd614d8d-ff6f-40c3-8ffb-e858ce38d1f9 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:41.827127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd614d8d-ff6f-40c3-8ffb-e858ce38d1f9 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:41.834104 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd10d8-14ec-4d86-974d-bc3ed8a07450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:41.835300 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd10d8-14ec-4d86-974d-bc3ed8a07450 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:41.835617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd10d8-14ec-4d86-974d-bc3ed8a07450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:41.835891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd10d8-14ec-4d86-974d-bc3ed8a07450 admin admin] PATH_INFO: `/v3/projects/954ddcd37bb24282912358ac246c2659/users/a8d27fc85d4b4fbca27149151bb4a10b/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:41.861886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd10d8-14ec-4d86-974d-bc3ed8a07450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=954ddcd37bb24282912358ac246c2659, user_id=a8d27fc85d4b4fbca27149151bb4a10b, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:41.865255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd10d8-14ec-4d86-974d-bc3ed8a07450 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:41.899037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1197/2408] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:33:41 2022] PUT /identity/v3/projects/954ddcd37bb24282912358ac246c2659/users/a8d27fc85d4b4fbca27149151bb4a10b/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:33:41.900728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdedd8-5322-4608-ab0d-3e367c2b4976 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:41.920588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abdedd8-5322-4608-ab0d-3e367c2b4976 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:42.002753 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:42.027196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a5586-4b08-4565-b4cc-0cb054d8c40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:42.029167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5586-4b08-4565-b4cc-0cb054d8c40e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:42.029167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5586-4b08-4565-b4cc-0cb054d8c40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:42.029167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5586-4b08-4565-b4cc-0cb054d8c40e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:42.030177 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a5586-4b08-4565-b4cc-0cb054d8c40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:42.031804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a5586-4b08-4565-b4cc-0cb054d8c40e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:42.062158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1212/2409] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:33:42.066709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd10d8-14ec-4d86-974d-bc3ed8a07450 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:42.081006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd10d8-14ec-4d86-974d-bc3ed8a07450 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:42.086566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bedc0-3464-4a10-bbb6-37dd19a9993d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:42.087091 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bedc0-3464-4a10-bbb6-37dd19a9993d admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:42.087310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bedc0-3464-4a10-bbb6-37dd19a9993d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:42.087606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bedc0-3464-4a10-bbb6-37dd19a9993d admin admin] PATH_INFO: `/v3/projects/954ddcd37bb24282912358ac246c2659/users/a8d27fc85d4b4fbca27149151bb4a10b/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:42.089753 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bedc0-3464-4a10-bbb6-37dd19a9993d admin admin] RBAC: Authorizing `identity:create_grant(project_id=954ddcd37bb24282912358ac246c2659, user_id=a8d27fc85d4b4fbca27149151bb4a10b, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:42.092501 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bedc0-3464-4a10-bbb6-37dd19a9993d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:42.109358 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1198/2410] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:33:42 2022] PUT /identity/v3/projects/954ddcd37bb24282912358ac246c2659/users/a8d27fc85d4b4fbca27149151bb4a10b/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:42.116608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c171e9e-22af-403b-9624-4ff8a34065c7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:42.116905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c171e9e-22af-403b-9624-4ff8a34065c7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:42.117173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c171e9e-22af-403b-9624-4ff8a34065c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:42.192417 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1c171e9e-22af-403b-9624-4ff8a34065c7 None None] MFA Rules not processed for user `a8d27fc85d4b4fbca27149151bb4a10b`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:42.193979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1c171e9e-22af-403b-9624-4ff8a34065c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:42.312222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1213/2411] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:42.326625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bedc0-3464-4a10-bbb6-37dd19a9993d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:42.330066 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-914bedc0-3464-4a10-bbb6-37dd19a9993d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:42.350661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:42.483337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bedc0-3464-4a10-bbb6-37dd19a9993d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:42.494280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1a0eb-3e1e-42d0-935b-c844622439a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:42.498078 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1a0eb-3e1e-42d0-935b-c844622439a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:42.498470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1a0eb-3e1e-42d0-935b-c844622439a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:42.498760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1a0eb-3e1e-42d0-935b-c844622439a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:42.520099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1a0eb-3e1e-42d0-935b-c844622439a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:42.521684 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1a0eb-3e1e-42d0-935b-c844622439a6 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:42.558050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1199/2412] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:42.691246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c171e9e-22af-403b-9624-4ff8a34065c7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:42.693118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1c171e9e-22af-403b-9624-4ff8a34065c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:42.708613 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c171e9e-22af-403b-9624-4ff8a34065c7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:42.713765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb2b8a-facc-4771-a50f-94446a3fcae1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:42.714999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb2b8a-facc-4771-a50f-94446a3fcae1 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:42.714999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb2b8a-facc-4771-a50f-94446a3fcae1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:42.715221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb2b8a-facc-4771-a50f-94446a3fcae1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:42.806948 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb2b8a-facc-4771-a50f-94446a3fcae1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:42.809007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb2b8a-facc-4771-a50f-94446a3fcae1 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:42.823811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1214/2413] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:43.658955 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1a0eb-3e1e-42d0-935b-c844622439a6 service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:43.660454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e5f1a0eb-3e1e-42d0-935b-c844622439a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:43.847849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1a0eb-3e1e-42d0-935b-c844622439a6 service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:43.858684 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaae67-8e19-4c42-8983-ee14f1d1faaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:43.859736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaae67-8e19-4c42-8983-ee14f1d1faaf service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:43.860169 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaae67-8e19-4c42-8983-ee14f1d1faaf service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:43.860358 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaae67-8e19-4c42-8983-ee14f1d1faaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:43.863145 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-19aaae67-8e19-4c42-8983-ee14f1d1faaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:43.878046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaae67-8e19-4c42-8983-ee14f1d1faaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:43.879762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaae67-8e19-4c42-8983-ee14f1d1faaf service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:43.893092 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1200/2414] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:45.805319 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3de27-e8ce-44e2-940f-ced28ca26b16 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:45.806281 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3de27-e8ce-44e2-940f-ced28ca26b16 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:45.806281 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3de27-e8ce-44e2-940f-ced28ca26b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:45.828026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e0b3de27-e8ce-44e2-940f-ced28ca26b16 None None] MFA Rules not processed for user `2d62232282dc4455834857d5290eb169`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:45.829403 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e0b3de27-e8ce-44e2-940f-ced28ca26b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:45.890995 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1215/2415] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:46.324608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaae67-8e19-4c42-8983-ee14f1d1faaf service nova] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:46.326051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-19aaae67-8e19-4c42-8983-ee14f1d1faaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:46.393146 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaae67-8e19-4c42-8983-ee14f1d1faaf service nova] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:46.400722 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643befa-ac9c-4224-ade8-5cabda5c3696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:46.401570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643befa-ac9c-4224-ade8-5cabda5c3696 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:46.402188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643befa-ac9c-4224-ade8-5cabda5c3696 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:46.402666 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643befa-ac9c-4224-ade8-5cabda5c3696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:46.405252 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c643befa-ac9c-4224-ade8-5cabda5c3696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:46.417157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643befa-ac9c-4224-ade8-5cabda5c3696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:46.418685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643befa-ac9c-4224-ade8-5cabda5c3696 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:46.433175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1201/2416] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:47.036256 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3de27-e8ce-44e2-940f-ced28ca26b16 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:47.037913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e0b3de27-e8ce-44e2-940f-ced28ca26b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:47.060700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:47.153779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3de27-e8ce-44e2-940f-ced28ca26b16 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:47.159305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cc219-f83d-4436-8d87-9fc9e9bc3c93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b3b0a0317b14e29a32eb6ba6b89c2aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:47.159957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cc219-f83d-4436-8d87-9fc9e9bc3c93 service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:47.160392 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cc219-f83d-4436-8d87-9fc9e9bc3c93 service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:47.160815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cc219-f83d-4436-8d87-9fc9e9bc3c93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:47.163435 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5b2cc219-f83d-4436-8d87-9fc9e9bc3c93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:47.177338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2cc219-f83d-4436-8d87-9fc9e9bc3c93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:47.179542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2cc219-f83d-4436-8d87-9fc9e9bc3c93 service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:47.190295 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1216/2417] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:50.605131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643befa-ac9c-4224-ade8-5cabda5c3696 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:50.614543 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643befa-ac9c-4224-ade8-5cabda5c3696 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:50.619019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92696c0-da40-4847-bfe3-9440c1eea072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:50.619697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92696c0-da40-4847-bfe3-9440c1eea072 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:50.619974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92696c0-da40-4847-bfe3-9440c1eea072 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:50.620244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92696c0-da40-4847-bfe3-9440c1eea072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:50.622040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d92696c0-da40-4847-bfe3-9440c1eea072 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:50.632295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92696c0-da40-4847-bfe3-9440c1eea072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:50.633746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92696c0-da40-4847-bfe3-9440c1eea072 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:50.643195 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1202/2418] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:52.702720 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cc219-f83d-4436-8d87-9fc9e9bc3c93 service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:52.705279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5b2cc219-f83d-4436-8d87-9fc9e9bc3c93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:52.727164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92696c0-da40-4847-bfe3-9440c1eea072 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:52.745603 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cc219-f83d-4436-8d87-9fc9e9bc3c93 service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:52.766326 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc0a1f-19f1-42d8-adee-dbc76b1d73a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:52.767221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc0a1f-19f1-42d8-adee-dbc76b1d73a5 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:52.767744 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc0a1f-19f1-42d8-adee-dbc76b1d73a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:52.768242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc0a1f-19f1-42d8-adee-dbc76b1d73a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:52.774422 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a2fc0a1f-19f1-42d8-adee-dbc76b1d73a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:52.775002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:52.797734 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc0a1f-19f1-42d8-adee-dbc76b1d73a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:52.805588 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc0a1f-19f1-42d8-adee-dbc76b1d73a5 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:52.829836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1217/2419] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:52.850654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92696c0-da40-4847-bfe3-9440c1eea072 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:52.857382 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e45ba1-fa07-47ee-86e8-ad10bf6482d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:52.857945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e45ba1-fa07-47ee-86e8-ad10bf6482d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:52.858410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e45ba1-fa07-47ee-86e8-ad10bf6482d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:52.858705 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e45ba1-fa07-47ee-86e8-ad10bf6482d4 admin admin] PATH_INFO: `/v3/users/4f74a7bc59f8444887a39720236e0adc` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:52.860529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e45ba1-fa07-47ee-86e8-ad10bf6482d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f74a7bc59f8444887a39720236e0adc)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:52.862940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e45ba1-fa07-47ee-86e8-ad10bf6482d4 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:52.962783 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-65e45ba1-fa07-47ee-86e8-ad10bf6482d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f74a7bc59f8444887a39720236e0adc'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:52.976736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-65e45ba1-fa07-47ee-86e8-ad10bf6482d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f74a7bc59f8444887a39720236e0adc'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:52.984610 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-65e45ba1-fa07-47ee-86e8-ad10bf6482d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f74a7bc59f8444887a39720236e0adc'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:52.985254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-65e45ba1-fa07-47ee-86e8-ad10bf6482d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f74a7bc59f8444887a39720236e0adc'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:52.993488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-65e45ba1-fa07-47ee-86e8-ad10bf6482d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f74a7bc59f8444887a39720236e0adc'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:52.999837 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1203/2420] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:33:52 2022] DELETE /identity/v3/users/4f74a7bc59f8444887a39720236e0adc => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:53.004223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fd365-23d7-4a1f-be83-d5f2eb63ebd4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:53.004573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fd365-23d7-4a1f-be83-d5f2eb63ebd4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:53.004912 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fd365-23d7-4a1f-be83-d5f2eb63ebd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:53.023494 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f02fd365-23d7-4a1f-be83-d5f2eb63ebd4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:53.024805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f02fd365-23d7-4a1f-be83-d5f2eb63ebd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:53.084836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1218/2421] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:33:53.094475 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e45ba1-fa07-47ee-86e8-ad10bf6482d4 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:53.097289 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-65e45ba1-fa07-47ee-86e8-ad10bf6482d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:53.194833 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e45ba1-fa07-47ee-86e8-ad10bf6482d4 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:53.200108 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbfa16-e0a4-46ce-bd63-96e8b95fb7f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:53.200933 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbfa16-e0a4-46ce-bd63-96e8b95fb7f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:53.201401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbfa16-e0a4-46ce-bd63-96e8b95fb7f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:53.201860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbfa16-e0a4-46ce-bd63-96e8b95fb7f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:53.212199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbfa16-e0a4-46ce-bd63-96e8b95fb7f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:53.213679 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbfa16-e0a4-46ce-bd63-96e8b95fb7f3 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:53.226424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1204/2422] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:53.691954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fd365-23d7-4a1f-be83-d5f2eb63ebd4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:53.694663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f02fd365-23d7-4a1f-be83-d5f2eb63ebd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:53.698038 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbfa16-e0a4-46ce-bd63-96e8b95fb7f3 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:53.699620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d5fbfa16-e0a4-46ce-bd63-96e8b95fb7f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:53.714044 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fd365-23d7-4a1f-be83-d5f2eb63ebd4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:53.728383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:53.729062 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:53.729342 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:53.729614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] PATH_INFO: `/v3/projects/f76f07f7718148c79260627f14f32b81` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:53.732901 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f76f07f7718148c79260627f14f32b81)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:53.737688 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:53.797395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:53.813632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f76f07f7718148c79260627f14f32b81', 'request_id': 'req-eec5b2ba-0b4c-4def-be0c-f91158394502'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:53.831252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f76f07f7718148c79260627f14f32b81', 'request_id': 'req-eec5b2ba-0b4c-4def-be0c-f91158394502'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:53.852793 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f76f07f7718148c79260627f14f32b81', 'request_id': 'req-eec5b2ba-0b4c-4def-be0c-f91158394502'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:53.861700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbfa16-e0a4-46ce-bd63-96e8b95fb7f3 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:53.883164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] The token cache is being invalidate because project f76f07f7718148c79260627f14f32b81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:33:53.883480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:53.885890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:53.888013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1219/2423] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:33:53 2022] DELETE /identity/v3/projects/f76f07f7718148c79260627f14f32b81 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:53.892090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:53.898706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:53.957821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:53.988360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8444f-2069-4fe9-b211-bfef00b8bff5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:53.989018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8444f-2069-4fe9-b211-bfef00b8bff5 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:53.989705 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8444f-2069-4fe9-b211-bfef00b8bff5 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:53.989989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8444f-2069-4fe9-b211-bfef00b8bff5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:53.994173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-63e8444f-2069-4fe9-b211-bfef00b8bff5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:54.010517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8444f-2069-4fe9-b211-bfef00b8bff5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:54.012764 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8444f-2069-4fe9-b211-bfef00b8bff5 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:54.027874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1205/2424] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:33:54.037262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5b2ba-0b4c-4def-be0c-f91158394502 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:54.047777 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6cbc1-ba4e-49ea-a7eb-eead5877cd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:54.048483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6cbc1-ba4e-49ea-a7eb-eead5877cd3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:54.048765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6cbc1-ba4e-49ea-a7eb-eead5877cd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:54.049042 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6cbc1-ba4e-49ea-a7eb-eead5877cd3a admin admin] PATH_INFO: `/v3/users/a8d27fc85d4b4fbca27149151bb4a10b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:54.050230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6cbc1-ba4e-49ea-a7eb-eead5877cd3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8d27fc85d4b4fbca27149151bb4a10b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:54.052343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6cbc1-ba4e-49ea-a7eb-eead5877cd3a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:54.119061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f1e6cbc1-ba4e-49ea-a7eb-eead5877cd3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8d27fc85d4b4fbca27149151bb4a10b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:54.127821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f1e6cbc1-ba4e-49ea-a7eb-eead5877cd3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8d27fc85d4b4fbca27149151bb4a10b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:54.142336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f1e6cbc1-ba4e-49ea-a7eb-eead5877cd3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8d27fc85d4b4fbca27149151bb4a10b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:54.153483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f1e6cbc1-ba4e-49ea-a7eb-eead5877cd3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8d27fc85d4b4fbca27149151bb4a10b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:54.154618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f1e6cbc1-ba4e-49ea-a7eb-eead5877cd3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8d27fc85d4b4fbca27149151bb4a10b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:54.162299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1220/2425] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:33:53 2022] DELETE /identity/v3/users/a8d27fc85d4b4fbca27149151bb4a10b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:33:54.779988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8444f-2069-4fe9-b211-bfef00b8bff5 service nova] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:54.809708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8444f-2069-4fe9-b211-bfef00b8bff5 service nova] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:54.880811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfddb6-c346-49ea-8f57-957797f2fadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:54.881566 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfddb6-c346-49ea-8f57-957797f2fadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:54.881916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfddb6-c346-49ea-8f57-957797f2fadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:54.882257 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfddb6-c346-49ea-8f57-957797f2fadd admin admin] PATH_INFO: `/v3/projects/954ddcd37bb24282912358ac246c2659` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:54.883686 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfddb6-c346-49ea-8f57-957797f2fadd admin admin] RBAC: Authorizing `identity:delete_project(project_id=954ddcd37bb24282912358ac246c2659)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:54.905077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfddb6-c346-49ea-8f57-957797f2fadd admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:54.962576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-41dfddb6-c346-49ea-8f57-957797f2fadd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '954ddcd37bb24282912358ac246c2659', 'request_id': 'req-41dfddb6-c346-49ea-8f57-957797f2fadd'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:54.968244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-41dfddb6-c346-49ea-8f57-957797f2fadd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '954ddcd37bb24282912358ac246c2659', 'request_id': 'req-41dfddb6-c346-49ea-8f57-957797f2fadd'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:54.976153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-41dfddb6-c346-49ea-8f57-957797f2fadd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '954ddcd37bb24282912358ac246c2659', 'request_id': 'req-41dfddb6-c346-49ea-8f57-957797f2fadd'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:54.983413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:54.988750 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-41dfddb6-c346-49ea-8f57-957797f2fadd admin admin] The token cache is being invalidate because project 954ddcd37bb24282912358ac246c2659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:33:54.989182 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-41dfddb6-c346-49ea-8f57-957797f2fadd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:54.990065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-41dfddb6-c346-49ea-8f57-957797f2fadd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:54.996083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1206/2426] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:33:54 2022] DELETE /identity/v3/projects/954ddcd37bb24282912358ac246c2659 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:54.997681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d762-d546-45f4-b022-fd42efe5a27c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:54.998143 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d762-d546-45f4-b022-fd42efe5a27c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:54.998303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d762-d546-45f4-b022-fd42efe5a27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:55.018951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-45e8d762-d546-45f4-b022-fd42efe5a27c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:55.020196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-45e8d762-d546-45f4-b022-fd42efe5a27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:55.082870 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1221/2427] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:55.092399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5995b32-5424-4f19-9075-62dfb83652e4 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:55.092689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5995b32-5424-4f19-9075-62dfb83652e4 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:55.092964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5995b32-5424-4f19-9075-62dfb83652e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:55.115681 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-a5995b32-5424-4f19-9075-62dfb83652e4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:55.116924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a5995b32-5424-4f19-9075-62dfb83652e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:55.126185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1207/2428] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:55.132380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8d762-d546-45f4-b022-fd42efe5a27c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:55.147507 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8d762-d546-45f4-b022-fd42efe5a27c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:55.152677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebfa04-8084-4534-8d15-bdafef71eddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:55.153285 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebfa04-8084-4534-8d15-bdafef71eddd admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:55.153566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebfa04-8084-4534-8d15-bdafef71eddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:55.153839 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebfa04-8084-4534-8d15-bdafef71eddd admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:55.154496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebfa04-8084-4534-8d15-bdafef71eddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:33:55.154798 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebfa04-8084-4534-8d15-bdafef71eddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:55.156451 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebfa04-8084-4534-8d15-bdafef71eddd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:55.169761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1222/2429] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:33:55.175099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5995b32-5424-4f19-9075-62dfb83652e4 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:55.185961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5995b32-5424-4f19-9075-62dfb83652e4 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:55.191124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139027f-911f-4397-bd73-a8e9a9a831bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:55.191908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139027f-911f-4397-bd73-a8e9a9a831bf admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:55.192289 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139027f-911f-4397-bd73-a8e9a9a831bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:55.192715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139027f-911f-4397-bd73-a8e9a9a831bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:55.194196 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139027f-911f-4397-bd73-a8e9a9a831bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:55.197020 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139027f-911f-4397-bd73-a8e9a9a831bf admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:55.237089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1208/2430] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:33:55 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:33:55.241608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebfa04-8084-4534-8d15-bdafef71eddd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:55.252586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebfa04-8084-4534-8d15-bdafef71eddd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:55.285784 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:55.340173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac3554-2962-4ac4-87d7-ad05f4160fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:55.341467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac3554-2962-4ac4-87d7-ad05f4160fff admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:55.341754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac3554-2962-4ac4-87d7-ad05f4160fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:55.342030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac3554-2962-4ac4-87d7-ad05f4160fff admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:55.342661 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac3554-2962-4ac4-87d7-ad05f4160fff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:55.346903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac3554-2962-4ac4-87d7-ad05f4160fff admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:55.380222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1223/2431] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:33:55 2022] POST /identity/v3/users => generated 283 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:33:55.384243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139027f-911f-4397-bd73-a8e9a9a831bf admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:55.401584 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139027f-911f-4397-bd73-a8e9a9a831bf admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:55.408079 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f911537-0cef-4253-b918-5fc5a5c8a9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:55.408830 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f911537-0cef-4253-b918-5fc5a5c8a9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:55.409395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f911537-0cef-4253-b918-5fc5a5c8a9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:55.409858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f911537-0cef-4253-b918-5fc5a5c8a9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:55.410691 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f911537-0cef-4253-b918-5fc5a5c8a9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:55.413789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f911537-0cef-4253-b918-5fc5a5c8a9f3 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:55.439890 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1209/2432] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:33:55.440699 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac3554-2962-4ac4-87d7-ad05f4160fff admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:55.453070 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac3554-2962-4ac4-87d7-ad05f4160fff admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:55.458801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f9c53-f5e4-4ad0-9f14-4f5f2aa210b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:55.460046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f9c53-f5e4-4ad0-9f14-4f5f2aa210b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:55.460327 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f9c53-f5e4-4ad0-9f14-4f5f2aa210b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:55.460598 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f9c53-f5e4-4ad0-9f14-4f5f2aa210b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:55.461247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f9c53-f5e4-4ad0-9f14-4f5f2aa210b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:55.463051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f9c53-f5e4-4ad0-9f14-4f5f2aa210b1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:55.477884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1224/2433] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:33:55.482234 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f911537-0cef-4253-b918-5fc5a5c8a9f3 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:55.497885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f911537-0cef-4253-b918-5fc5a5c8a9f3 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:55.505081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01c257-4830-4952-b3ac-c601adeb1068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:55.506707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01c257-4830-4952-b3ac-c601adeb1068 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:55.507034 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01c257-4830-4952-b3ac-c601adeb1068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:55.507544 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01c257-4830-4952-b3ac-c601adeb1068 admin admin] PATH_INFO: `/v3/projects/19b1e8953a424a23bb91b6779f6d392b/users/c9712a612a864bcc8cffda772ec5ad3a/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:55.539929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01c257-4830-4952-b3ac-c601adeb1068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b1e8953a424a23bb91b6779f6d392b, user_id=c9712a612a864bcc8cffda772ec5ad3a, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:55.544247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01c257-4830-4952-b3ac-c601adeb1068 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:55.567649 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1210/2434] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:33:55 2022] PUT /identity/v3/projects/19b1e8953a424a23bb91b6779f6d392b/users/c9712a612a864bcc8cffda772ec5ad3a/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:55.574030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ff881-33b5-43fd-b34d-3c1731cf847f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:55.574730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ff881-33b5-43fd-b34d-3c1731cf847f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:55.575227 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ff881-33b5-43fd-b34d-3c1731cf847f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:55.608178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0a0ff881-33b5-43fd-b34d-3c1731cf847f None None] MFA Rules not processed for user `c9712a612a864bcc8cffda772ec5ad3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:55.609406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0a0ff881-33b5-43fd-b34d-3c1731cf847f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:55.676017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1225/2435] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:55.680518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01c257-4830-4952-b3ac-c601adeb1068 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:55.749472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01c257-4830-4952-b3ac-c601adeb1068 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:55.754275 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257f7d5-b5b0-4a81-b578-cc7589f9f95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:55.754904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257f7d5-b5b0-4a81-b578-cc7589f9f95b admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:55.755180 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257f7d5-b5b0-4a81-b578-cc7589f9f95b admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:55.755449 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257f7d5-b5b0-4a81-b578-cc7589f9f95b admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:55.756692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257f7d5-b5b0-4a81-b578-cc7589f9f95b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:55.758580 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257f7d5-b5b0-4a81-b578-cc7589f9f95b admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:55.793699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1211/2436] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:33:55 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:33:55.795122 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ff881-33b5-43fd-b34d-3c1731cf847f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:55.807032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ff881-33b5-43fd-b34d-3c1731cf847f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:55.878946 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5ee6e-2dbf-41f7-80da-de047d9015bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:55.879704 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5ee6e-2dbf-41f7-80da-de047d9015bf admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:55.880086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5ee6e-2dbf-41f7-80da-de047d9015bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:55.880463 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5ee6e-2dbf-41f7-80da-de047d9015bf admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:55.881734 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5ee6e-2dbf-41f7-80da-de047d9015bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:55.883889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5ee6e-2dbf-41f7-80da-de047d9015bf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:55.911932 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1226/2437] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:33:55 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:33:55.914081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257f7d5-b5b0-4a81-b578-cc7589f9f95b admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:55.928277 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257f7d5-b5b0-4a81-b578-cc7589f9f95b admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:55.935719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e45fc-0fc2-4387-b3aa-c0d4bc98a68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:55.936448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e45fc-0fc2-4387-b3aa-c0d4bc98a68d admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:55.937202 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e45fc-0fc2-4387-b3aa-c0d4bc98a68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:55.937614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e45fc-0fc2-4387-b3aa-c0d4bc98a68d admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:55.938404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e45fc-0fc2-4387-b3aa-c0d4bc98a68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:55.940338 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e45fc-0fc2-4387-b3aa-c0d4bc98a68d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:55.955046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1212/2438] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:33:55.959859 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5ee6e-2dbf-41f7-80da-de047d9015bf admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:55.972147 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5ee6e-2dbf-41f7-80da-de047d9015bf admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:55.979188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b80d3a-6133-41dc-80bf-fb78d99abdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:55.980301 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b80d3a-6133-41dc-80bf-fb78d99abdf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:55.980594 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b80d3a-6133-41dc-80bf-fb78d99abdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:55.980918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b80d3a-6133-41dc-80bf-fb78d99abdf7 admin admin] PATH_INFO: `/v3/projects/6e28c0aeaf4e4b788d9bbb8f7f1c05bc/users/f4aaae207523472f9ea1527976d28ca8/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:55.996526 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b80d3a-6133-41dc-80bf-fb78d99abdf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e28c0aeaf4e4b788d9bbb8f7f1c05bc, user_id=f4aaae207523472f9ea1527976d28ca8, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:55.999350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b80d3a-6133-41dc-80bf-fb78d99abdf7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:56.011517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:56.018306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1227/2439] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:33:55 2022] PUT /identity/v3/projects/6e28c0aeaf4e4b788d9bbb8f7f1c05bc/users/f4aaae207523472f9ea1527976d28ca8/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:56.022937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e45fc-0fc2-4387-b3aa-c0d4bc98a68d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:56.042950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e45fc-0fc2-4387-b3aa-c0d4bc98a68d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:56.119622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d347f-b7cd-475c-967f-ef25e2401541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:56.120290 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d347f-b7cd-475c-967f-ef25e2401541 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:56.120575 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d347f-b7cd-475c-967f-ef25e2401541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:56.120845 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d347f-b7cd-475c-967f-ef25e2401541 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:56.121620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d347f-b7cd-475c-967f-ef25e2401541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:56.123537 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d347f-b7cd-475c-967f-ef25e2401541 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:56.138073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1213/2440] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:33:56.139722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b80d3a-6133-41dc-80bf-fb78d99abdf7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:56.151219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b80d3a-6133-41dc-80bf-fb78d99abdf7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:56.156171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89f628-cc02-467b-852e-f707c090c823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:56.156979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89f628-cc02-467b-852e-f707c090c823 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:56.157374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89f628-cc02-467b-852e-f707c090c823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:56.157885 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89f628-cc02-467b-852e-f707c090c823 admin admin] PATH_INFO: `/v3/projects/6e28c0aeaf4e4b788d9bbb8f7f1c05bc/users/f4aaae207523472f9ea1527976d28ca8/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:56.160292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89f628-cc02-467b-852e-f707c090c823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e28c0aeaf4e4b788d9bbb8f7f1c05bc, user_id=f4aaae207523472f9ea1527976d28ca8, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:56.163018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89f628-cc02-467b-852e-f707c090c823 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:56.175121 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1228/2441] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:33:56 2022] PUT /identity/v3/projects/6e28c0aeaf4e4b788d9bbb8f7f1c05bc/users/f4aaae207523472f9ea1527976d28ca8/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:56.187448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c802ee-33d1-484a-9b49-a214508c0f51 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:56.187831 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c802ee-33d1-484a-9b49-a214508c0f51 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:56.188102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c802ee-33d1-484a-9b49-a214508c0f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:56.202170 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:56.224755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-72c802ee-33d1-484a-9b49-a214508c0f51 None None] MFA Rules not processed for user `f4aaae207523472f9ea1527976d28ca8`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:56.225989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-72c802ee-33d1-484a-9b49-a214508c0f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:56.296009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1214/2442] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:56.307626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89f628-cc02-467b-852e-f707c090c823 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:56.309323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0c89f628-cc02-467b-852e-f707c090c823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:56.369995 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89f628-cc02-467b-852e-f707c090c823 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:56.373591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eea4d6-a4d6-4605-b168-1a50fd2b15f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:56.374226 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eea4d6-a4d6-4605-b168-1a50fd2b15f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:56.374506 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eea4d6-a4d6-4605-b168-1a50fd2b15f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:56.374780 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eea4d6-a4d6-4605-b168-1a50fd2b15f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:56.384155 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eea4d6-a4d6-4605-b168-1a50fd2b15f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:56.385568 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eea4d6-a4d6-4605-b168-1a50fd2b15f7 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:56.394809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1229/2443] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:56.468257 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c802ee-33d1-484a-9b49-a214508c0f51 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:56.477999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c802ee-33d1-484a-9b49-a214508c0f51 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:56.481605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d794769-c173-4339-8c62-7e4275be357f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:56.482204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d794769-c173-4339-8c62-7e4275be357f service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:56.482486 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d794769-c173-4339-8c62-7e4275be357f service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:56.482763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d794769-c173-4339-8c62-7e4275be357f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:56.540811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d794769-c173-4339-8c62-7e4275be357f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:56.542257 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d794769-c173-4339-8c62-7e4275be357f service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:56.551349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1215/2444] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:33:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:33:58.433426 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4a354-5240-4e92-aa12-bdfc3da38990 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:58.433889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4a354-5240-4e92-aa12-bdfc3da38990 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:58.434006 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4a354-5240-4e92-aa12-bdfc3da38990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:58.454795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e5e4a354-5240-4e92-aa12-bdfc3da38990 None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:58.456038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e5e4a354-5240-4e92-aa12-bdfc3da38990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:58.526157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1230/2445] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:33:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:33:58.538307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d794769-c173-4339-8c62-7e4275be357f service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:58.539854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9d794769-c173-4339-8c62-7e4275be357f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:58.634242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d794769-c173-4339-8c62-7e4275be357f service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:58.642234 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa38c1b-1c2b-4779-8b0e-f9389d991854 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:58.643132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa38c1b-1c2b-4779-8b0e-f9389d991854 service swift] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:58.643596 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa38c1b-1c2b-4779-8b0e-f9389d991854 service swift] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:58.644072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa38c1b-1c2b-4779-8b0e-f9389d991854 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:58.654226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa38c1b-1c2b-4779-8b0e-f9389d991854 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:58.656102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa38c1b-1c2b-4779-8b0e-f9389d991854 service swift] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:58.665777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1216/2446] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:33:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:33:59.433565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4a354-5240-4e92-aa12-bdfc3da38990 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:33:59.534874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4a354-5240-4e92-aa12-bdfc3da38990 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:33:59.543051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de1ba8-ec79-4c39-9300-7e07a1a71776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:33:59.544202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de1ba8-ec79-4c39-9300-7e07a1a71776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:59.544644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de1ba8-ec79-4c39-9300-7e07a1a71776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:59.545544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de1ba8-ec79-4c39-9300-7e07a1a71776 admin admin] PATH_INFO: `/v3/users/c9712a612a864bcc8cffda772ec5ad3a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:59.549146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de1ba8-ec79-4c39-9300-7e07a1a71776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9712a612a864bcc8cffda772ec5ad3a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:33:59.551348 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de1ba8-ec79-4c39-9300-7e07a1a71776 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:33:59.656249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:33:59.675953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4de1ba8-ec79-4c39-9300-7e07a1a71776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9712a612a864bcc8cffda772ec5ad3a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:59.697956 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4de1ba8-ec79-4c39-9300-7e07a1a71776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9712a612a864bcc8cffda772ec5ad3a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:59.736395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4de1ba8-ec79-4c39-9300-7e07a1a71776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9712a612a864bcc8cffda772ec5ad3a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:59.761038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4de1ba8-ec79-4c39-9300-7e07a1a71776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9712a612a864bcc8cffda772ec5ad3a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:59.762137 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4de1ba8-ec79-4c39-9300-7e07a1a71776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9712a612a864bcc8cffda772ec5ad3a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:33:59.769197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1231/2447] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:33:59 2022] DELETE /identity/v3/users/c9712a612a864bcc8cffda772ec5ad3a => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:33:59.774406 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ba749-3d5f-45cf-b8c1-3779ba583493 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:33:59.774952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ba749-3d5f-45cf-b8c1-3779ba583493 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:33:59.775439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ba749-3d5f-45cf-b8c1-3779ba583493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:33:59.843130 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-f12ba749-3d5f-45cf-b8c1-3779ba583493 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:33:59.844599 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-f12ba749-3d5f-45cf-b8c1-3779ba583493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:33:59.902371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:00.054949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1217/2448] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:00.060877 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de1ba8-ec79-4c39-9300-7e07a1a71776 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:00.062492 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d4de1ba8-ec79-4c39-9300-7e07a1a71776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:00.162977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de1ba8-ec79-4c39-9300-7e07a1a71776 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:00.173196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9650c7-6200-4afe-870a-4d303dedc1f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:00.174299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9650c7-6200-4afe-870a-4d303dedc1f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:00.174858 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9650c7-6200-4afe-870a-4d303dedc1f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:00.175373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9650c7-6200-4afe-870a-4d303dedc1f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:00.192450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9650c7-6200-4afe-870a-4d303dedc1f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:00.194477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9650c7-6200-4afe-870a-4d303dedc1f1 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:00.220136 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1232/2449] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:34:00.618293 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ba749-3d5f-45cf-b8c1-3779ba583493 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:00.620191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-f12ba749-3d5f-45cf-b8c1-3779ba583493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:00.642079 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ba749-3d5f-45cf-b8c1-3779ba583493 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:00.654514 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:00.655180 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:00.655464 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:00.655849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] PATH_INFO: `/v3/projects/19b1e8953a424a23bb91b6779f6d392b` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:00.658845 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b1e8953a424a23bb91b6779f6d392b)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:00.660984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:00.751777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b1e8953a424a23bb91b6779f6d392b', 'request_id': 'req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:00.761081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b1e8953a424a23bb91b6779f6d392b', 'request_id': 'req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:00.770739 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b1e8953a424a23bb91b6779f6d392b', 'request_id': 'req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:00.787887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] The token cache is being invalidate because project 19b1e8953a424a23bb91b6779f6d392b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:00.788544 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:00.789544 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:00.794450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1218/2450] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:00 2022] DELETE /identity/v3/projects/19b1e8953a424a23bb91b6779f6d392b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:00.795626 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9650c7-6200-4afe-870a-4d303dedc1f1 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:00.820689 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9650c7-6200-4afe-870a-4d303dedc1f1 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:00.902342 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0eb0a-d48c-4ddd-8e99-b205c19270cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:00.902980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0eb0a-d48c-4ddd-8e99-b205c19270cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:00.903263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0eb0a-d48c-4ddd-8e99-b205c19270cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:00.903543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0eb0a-d48c-4ddd-8e99-b205c19270cf admin admin] PATH_INFO: `/v3/users/f4aaae207523472f9ea1527976d28ca8` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:00.904700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0eb0a-d48c-4ddd-8e99-b205c19270cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4aaae207523472f9ea1527976d28ca8)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:00.906632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0eb0a-d48c-4ddd-8e99-b205c19270cf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:01.022132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eee0eb0a-d48c-4ddd-8e99-b205c19270cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4aaae207523472f9ea1527976d28ca8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:01.036292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eee0eb0a-d48c-4ddd-8e99-b205c19270cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4aaae207523472f9ea1527976d28ca8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:01.044691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:01.062909 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eee0eb0a-d48c-4ddd-8e99-b205c19270cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4aaae207523472f9ea1527976d28ca8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:01.073176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eee0eb0a-d48c-4ddd-8e99-b205c19270cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4aaae207523472f9ea1527976d28ca8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:01.075206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eee0eb0a-d48c-4ddd-8e99-b205c19270cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4aaae207523472f9ea1527976d28ca8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:01.085690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1233/2451] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:00 2022] DELETE /identity/v3/users/f4aaae207523472f9ea1527976d28ca8 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:01.488769 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:01.490356 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:01.518487 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad68f45-07c9-4729-a0f2-02ff0f8f0d52 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:01.593250 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:01.593900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:01.594186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:01.594463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8 admin admin] PATH_INFO: `/v3/projects/6e28c0aeaf4e4b788d9bbb8f7f1c05bc` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:01.595634 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e28c0aeaf4e4b788d9bbb8f7f1c05bc)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:01.597601 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:01.647426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:01.651825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e28c0aeaf4e4b788d9bbb8f7f1c05bc', 'request_id': 'req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:01.659020 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e28c0aeaf4e4b788d9bbb8f7f1c05bc', 'request_id': 'req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:01.668618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e28c0aeaf4e4b788d9bbb8f7f1c05bc', 'request_id': 'req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:01.681880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8 admin admin] The token cache is being invalidate because project 6e28c0aeaf4e4b788d9bbb8f7f1c05bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:01.682188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:01.682843 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-c1c55f9b-c2ae-43df-ab5d-0b64d55afca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:01.684673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1219/2452] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:01 2022] DELETE /identity/v3/projects/6e28c0aeaf4e4b788d9bbb8f7f1c05bc => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:01.689602 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fffd9-eacf-4d19-aa26-e3ed2c9fa5e6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:01.689922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fffd9-eacf-4d19-aa26-e3ed2c9fa5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:01.690197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fffd9-eacf-4d19-aa26-e3ed2c9fa5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:01.711344 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-360fffd9-eacf-4d19-aa26-e3ed2c9fa5e6 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:01.712630 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-360fffd9-eacf-4d19-aa26-e3ed2c9fa5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:01.775404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1234/2453] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:01.783851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3deabb-1ee0-41bf-90e2-471ecd7f4419 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:01.784195 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3deabb-1ee0-41bf-90e2-471ecd7f4419 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:01.784564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3deabb-1ee0-41bf-90e2-471ecd7f4419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:01.805958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-9b3deabb-1ee0-41bf-90e2-471ecd7f4419 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:01.808399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9b3deabb-1ee0-41bf-90e2-471ecd7f4419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:01.815460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1220/2454] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:01.820341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fffd9-eacf-4d19-aa26-e3ed2c9fa5e6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:01.830373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fffd9-eacf-4d19-aa26-e3ed2c9fa5e6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:01.838573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e8b88-7e8f-405f-b8f1-7a693e22ca50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:01.839206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e8b88-7e8f-405f-b8f1-7a693e22ca50 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:01.839489 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e8b88-7e8f-405f-b8f1-7a693e22ca50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:01.839794 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e8b88-7e8f-405f-b8f1-7a693e22ca50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:01.840518 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e8b88-7e8f-405f-b8f1-7a693e22ca50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:34:01.840819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e8b88-7e8f-405f-b8f1-7a693e22ca50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:01.842825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e8b88-7e8f-405f-b8f1-7a693e22ca50 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:01.855740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1235/2455] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:34:01.859884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3deabb-1ee0-41bf-90e2-471ecd7f4419 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:01.869281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3deabb-1ee0-41bf-90e2-471ecd7f4419 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:01.874339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853da13-22d4-4b69-a1fb-9c730b4e1602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:01.874953 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853da13-22d4-4b69-a1fb-9c730b4e1602 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:01.875235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853da13-22d4-4b69-a1fb-9c730b4e1602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:01.875509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853da13-22d4-4b69-a1fb-9c730b4e1602 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:01.876205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c853da13-22d4-4b69-a1fb-9c730b4e1602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:01.878120 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c853da13-22d4-4b69-a1fb-9c730b4e1602 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:01.918718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1221/2456] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:01 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:01.920000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e8b88-7e8f-405f-b8f1-7a693e22ca50 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:01.938028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e8b88-7e8f-405f-b8f1-7a693e22ca50 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:02.016078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf604dd-5cc3-4bf8-8833-1e4a27e75bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:02.016958 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf604dd-5cc3-4bf8-8833-1e4a27e75bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:02.017331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf604dd-5cc3-4bf8-8833-1e4a27e75bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:02.017682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf604dd-5cc3-4bf8-8833-1e4a27e75bd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:02.018465 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf604dd-5cc3-4bf8-8833-1e4a27e75bd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:02.020549 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf604dd-5cc3-4bf8-8833-1e4a27e75bd1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:02.052148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1236/2457] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:01 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:02.056037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853da13-22d4-4b69-a1fb-9c730b4e1602 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:02.067755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853da13-22d4-4b69-a1fb-9c730b4e1602 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:02.073396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ff4ac-b0b3-4b6b-b7ac-84551356515f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:02.074093 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ff4ac-b0b3-4b6b-b7ac-84551356515f admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:02.074466 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ff4ac-b0b3-4b6b-b7ac-84551356515f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:02.074822 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ff4ac-b0b3-4b6b-b7ac-84551356515f admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:02.075708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ff4ac-b0b3-4b6b-b7ac-84551356515f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:02.077412 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ff4ac-b0b3-4b6b-b7ac-84551356515f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:02.098591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1222/2458] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:02.104191 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf604dd-5cc3-4bf8-8833-1e4a27e75bd1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:02.120566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf604dd-5cc3-4bf8-8833-1e4a27e75bd1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:02.129360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7186df9-47d1-41b7-be69-7673bc283fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:02.130128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7186df9-47d1-41b7-be69-7673bc283fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:02.130990 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7186df9-47d1-41b7-be69-7673bc283fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:02.131262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7186df9-47d1-41b7-be69-7673bc283fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:02.131966 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7186df9-47d1-41b7-be69-7673bc283fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:02.146873 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7186df9-47d1-41b7-be69-7673bc283fe3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:02.168139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1237/2459] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:34:02.169347 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ff4ac-b0b3-4b6b-b7ac-84551356515f admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:02.181978 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ff4ac-b0b3-4b6b-b7ac-84551356515f admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:02.201640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ec048-c9b7-42c7-b9f0-4b65c237aaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:02.202322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ec048-c9b7-42c7-b9f0-4b65c237aaf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:02.202604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ec048-c9b7-42c7-b9f0-4b65c237aaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:02.202880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ec048-c9b7-42c7-b9f0-4b65c237aaf2 admin admin] PATH_INFO: `/v3/projects/2c242df16aef4d0080a05a74a9ff7e2c/users/667afa6673d44733a0db67003ff3f618/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:02.219658 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ec048-c9b7-42c7-b9f0-4b65c237aaf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c242df16aef4d0080a05a74a9ff7e2c, user_id=667afa6673d44733a0db67003ff3f618, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:02.222315 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ec048-c9b7-42c7-b9f0-4b65c237aaf2 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:02.243028 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1223/2460] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:02 2022] PUT /identity/v3/projects/2c242df16aef4d0080a05a74a9ff7e2c/users/667afa6673d44733a0db67003ff3f618/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:02.253216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa22d1-aa69-4358-aa3b-28aabfeeeaf5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:02.253573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa22d1-aa69-4358-aa3b-28aabfeeeaf5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:02.253855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa22d1-aa69-4358-aa3b-28aabfeeeaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:02.284067 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dcaa22d1-aa69-4358-aa3b-28aabfeeeaf5 None None] MFA Rules not processed for user `667afa6673d44733a0db67003ff3f618`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:02.285493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dcaa22d1-aa69-4358-aa3b-28aabfeeeaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:02.325303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:02.381673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1238/2461] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:02.386130 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ec048-c9b7-42c7-b9f0-4b65c237aaf2 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:02.481995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ec048-c9b7-42c7-b9f0-4b65c237aaf2 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:02.487412 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01081d-6261-4fd3-bb96-4b4728e9398c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:02.488153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01081d-6261-4fd3-bb96-4b4728e9398c admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:02.489242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01081d-6261-4fd3-bb96-4b4728e9398c admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:02.489638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01081d-6261-4fd3-bb96-4b4728e9398c admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:02.490413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01081d-6261-4fd3-bb96-4b4728e9398c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:02.492455 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01081d-6261-4fd3-bb96-4b4728e9398c admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:02.523889 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1224/2462] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:02 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:02.525817 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa22d1-aa69-4358-aa3b-28aabfeeeaf5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:02.538423 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa22d1-aa69-4358-aa3b-28aabfeeeaf5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:02.608643 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c223f34-c9ce-4e3e-a97c-491c868ac2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:02.609632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c223f34-c9ce-4e3e-a97c-491c868ac2f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:02.610087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c223f34-c9ce-4e3e-a97c-491c868ac2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:02.610690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c223f34-c9ce-4e3e-a97c-491c868ac2f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:02.611661 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c223f34-c9ce-4e3e-a97c-491c868ac2f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:02.613943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c223f34-c9ce-4e3e-a97c-491c868ac2f3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:02.641868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1239/2463] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:02 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:02.645574 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01081d-6261-4fd3-bb96-4b4728e9398c admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:02.658060 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01081d-6261-4fd3-bb96-4b4728e9398c admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:02.663256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f650e-16ad-495a-a4ec-18753c0d2a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:02.663922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f650e-16ad-495a-a4ec-18753c0d2a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:02.664329 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f650e-16ad-495a-a4ec-18753c0d2a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:02.664617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f650e-16ad-495a-a4ec-18753c0d2a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:02.665261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f650e-16ad-495a-a4ec-18753c0d2a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:02.666872 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f650e-16ad-495a-a4ec-18753c0d2a23 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:02.678221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1225/2464] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:02.682535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c223f34-c9ce-4e3e-a97c-491c868ac2f3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:02.694661 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c223f34-c9ce-4e3e-a97c-491c868ac2f3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:02.699949 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5dda30-4bc3-4d1a-aeb1-7c2621611eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:02.700698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dda30-4bc3-4d1a-aeb1-7c2621611eea admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:02.700979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dda30-4bc3-4d1a-aeb1-7c2621611eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:02.701252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dda30-4bc3-4d1a-aeb1-7c2621611eea admin admin] PATH_INFO: `/v3/projects/d0ff4a58b25e4d3492c25c1251c1d3f3/users/51aeb065f7df44ce9bae3db054130669/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:02.718508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5dda30-4bc3-4d1a-aeb1-7c2621611eea admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ff4a58b25e4d3492c25c1251c1d3f3, user_id=51aeb065f7df44ce9bae3db054130669, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:02.721203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5dda30-4bc3-4d1a-aeb1-7c2621611eea admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:02.735537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1240/2465] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:02 2022] PUT /identity/v3/projects/d0ff4a58b25e4d3492c25c1251c1d3f3/users/51aeb065f7df44ce9bae3db054130669/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:02.739411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f650e-16ad-495a-a4ec-18753c0d2a23 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:02.749690 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f650e-16ad-495a-a4ec-18753c0d2a23 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:02.791895 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:02.816116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a41ab-2c6b-4ba5-8d4e-70f1309aa0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:02.816777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a41ab-2c6b-4ba5-8d4e-70f1309aa0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:02.817305 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a41ab-2c6b-4ba5-8d4e-70f1309aa0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:02.817680 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a41ab-2c6b-4ba5-8d4e-70f1309aa0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:02.818962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a41ab-2c6b-4ba5-8d4e-70f1309aa0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:02.820776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a41ab-2c6b-4ba5-8d4e-70f1309aa0e6 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:02.832319 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1226/2466] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:02.837553 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5dda30-4bc3-4d1a-aeb1-7c2621611eea admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:02.848824 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5dda30-4bc3-4d1a-aeb1-7c2621611eea admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:02.853666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dce4ab-ed0c-481c-9856-0fbac500a5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:02.854328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dce4ab-ed0c-481c-9856-0fbac500a5db admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:02.854616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dce4ab-ed0c-481c-9856-0fbac500a5db admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:02.854889 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dce4ab-ed0c-481c-9856-0fbac500a5db admin admin] PATH_INFO: `/v3/projects/d0ff4a58b25e4d3492c25c1251c1d3f3/users/51aeb065f7df44ce9bae3db054130669/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:02.856864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dce4ab-ed0c-481c-9856-0fbac500a5db admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ff4a58b25e4d3492c25c1251c1d3f3, user_id=51aeb065f7df44ce9bae3db054130669, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:02.859532 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dce4ab-ed0c-481c-9856-0fbac500a5db admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:02.870812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1241/2467] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:02 2022] PUT /identity/v3/projects/d0ff4a58b25e4d3492c25c1251c1d3f3/users/51aeb065f7df44ce9bae3db054130669/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:02.882353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e89ad-902a-4f5f-9dbe-dba8f8a41624 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:02.882646 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e89ad-902a-4f5f-9dbe-dba8f8a41624 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:02.883058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e89ad-902a-4f5f-9dbe-dba8f8a41624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:02.918049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-334e89ad-902a-4f5f-9dbe-dba8f8a41624 None None] MFA Rules not processed for user `51aeb065f7df44ce9bae3db054130669`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:02.919315 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-334e89ad-902a-4f5f-9dbe-dba8f8a41624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:02.995949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1227/2468] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:03.008260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dce4ab-ed0c-481c-9856-0fbac500a5db admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:03.009980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b9dce4ab-ed0c-481c-9856-0fbac500a5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:03.075278 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dce4ab-ed0c-481c-9856-0fbac500a5db admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:03.079104 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07333ff7-dc41-485d-8b81-48a099c51a48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:03.079755 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07333ff7-dc41-485d-8b81-48a099c51a48 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:03.080063 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07333ff7-dc41-485d-8b81-48a099c51a48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:03.080371 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07333ff7-dc41-485d-8b81-48a099c51a48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:03.090025 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07333ff7-dc41-485d-8b81-48a099c51a48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:03.091473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07333ff7-dc41-485d-8b81-48a099c51a48 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:03.102087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1242/2469] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:03.136510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e89ad-902a-4f5f-9dbe-dba8f8a41624 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:03.224280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e89ad-902a-4f5f-9dbe-dba8f8a41624 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:03.231934 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf63b3-be29-4a68-aba4-f8a8fc50e8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:03.232677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf63b3-be29-4a68-aba4-f8a8fc50e8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:03.232960 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf63b3-be29-4a68-aba4-f8a8fc50e8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:03.233239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf63b3-be29-4a68-aba4-f8a8fc50e8fa admin admin] PATH_INFO: `/v3/users/667afa6673d44733a0db67003ff3f618` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:03.234404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf63b3-be29-4a68-aba4-f8a8fc50e8fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=667afa6673d44733a0db67003ff3f618)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:03.236875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf63b3-be29-4a68-aba4-f8a8fc50e8fa admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:03.327704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9acf63b3-be29-4a68-aba4-f8a8fc50e8fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '667afa6673d44733a0db67003ff3f618'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:03.341715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9acf63b3-be29-4a68-aba4-f8a8fc50e8fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '667afa6673d44733a0db67003ff3f618'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:03.343858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:03.352487 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9acf63b3-be29-4a68-aba4-f8a8fc50e8fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '667afa6673d44733a0db67003ff3f618'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:03.355806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9acf63b3-be29-4a68-aba4-f8a8fc50e8fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '667afa6673d44733a0db67003ff3f618'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:03.367444 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9acf63b3-be29-4a68-aba4-f8a8fc50e8fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '667afa6673d44733a0db67003ff3f618'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:03.374439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1228/2470] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:03 2022] DELETE /identity/v3/users/667afa6673d44733a0db67003ff3f618 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:03.379055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729152d-ba01-4852-99cc-6ddbf6d84a50 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:03.379399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729152d-ba01-4852-99cc-6ddbf6d84a50 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:03.379702 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729152d-ba01-4852-99cc-6ddbf6d84a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:03.406185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2729152d-ba01-4852-99cc-6ddbf6d84a50 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:03.407493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2729152d-ba01-4852-99cc-6ddbf6d84a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:03.430042 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:03.483281 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1243/2471] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:03.494609 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf63b3-be29-4a68-aba4-f8a8fc50e8fa admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:03.501249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9acf63b3-be29-4a68-aba4-f8a8fc50e8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:03.663366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf63b3-be29-4a68-aba4-f8a8fc50e8fa admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:03.675601 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b6e42-fa3d-47d1-b4fe-1c942fdf2dbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:03.676592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b6e42-fa3d-47d1-b4fe-1c942fdf2dbe service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:03.677068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b6e42-fa3d-47d1-b4fe-1c942fdf2dbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:03.678277 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b6e42-fa3d-47d1-b4fe-1c942fdf2dbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:03.702776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b6e42-fa3d-47d1-b4fe-1c942fdf2dbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:03.705039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b6e42-fa3d-47d1-b4fe-1c942fdf2dbe service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:03.718790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1229/2472] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:04.169726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729152d-ba01-4852-99cc-6ddbf6d84a50 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:04.169726 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2729152d-ba01-4852-99cc-6ddbf6d84a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:04.188078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729152d-ba01-4852-99cc-6ddbf6d84a50 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:04.193460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:04.194330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:04.194944 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:04.195320 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] PATH_INFO: `/v3/projects/2c242df16aef4d0080a05a74a9ff7e2c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:04.196773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c242df16aef4d0080a05a74a9ff7e2c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:04.200865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:04.257498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c242df16aef4d0080a05a74a9ff7e2c', 'request_id': 'req-5530b72a-02c5-44f4-8061-d65cf1970418'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:04.265291 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c242df16aef4d0080a05a74a9ff7e2c', 'request_id': 'req-5530b72a-02c5-44f4-8061-d65cf1970418'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:04.274087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c242df16aef4d0080a05a74a9ff7e2c', 'request_id': 'req-5530b72a-02c5-44f4-8061-d65cf1970418'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:04.290171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] The token cache is being invalidate because project 2c242df16aef4d0080a05a74a9ff7e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:04.290463 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:04.291150 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:04.292666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1244/2473] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:04 2022] DELETE /identity/v3/projects/2c242df16aef4d0080a05a74a9ff7e2c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:04.296798 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b6e42-fa3d-47d1-b4fe-1c942fdf2dbe service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:04.308066 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b6e42-fa3d-47d1-b4fe-1c942fdf2dbe service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:04.392740 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8cffe-6767-4028-a7c9-fa050a9bf275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:04.393447 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8cffe-6767-4028-a7c9-fa050a9bf275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:04.393734 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8cffe-6767-4028-a7c9-fa050a9bf275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:04.394006 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8cffe-6767-4028-a7c9-fa050a9bf275 admin admin] PATH_INFO: `/v3/users/51aeb065f7df44ce9bae3db054130669` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:04.395153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8cffe-6767-4028-a7c9-fa050a9bf275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51aeb065f7df44ce9bae3db054130669)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:04.398232 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8cffe-6767-4028-a7c9-fa050a9bf275 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:04.478020 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7ba8cffe-6767-4028-a7c9-fa050a9bf275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51aeb065f7df44ce9bae3db054130669'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:04.498505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7ba8cffe-6767-4028-a7c9-fa050a9bf275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51aeb065f7df44ce9bae3db054130669'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:04.531446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7ba8cffe-6767-4028-a7c9-fa050a9bf275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51aeb065f7df44ce9bae3db054130669'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:04.532357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7ba8cffe-6767-4028-a7c9-fa050a9bf275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51aeb065f7df44ce9bae3db054130669'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:04.549805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7ba8cffe-6767-4028-a7c9-fa050a9bf275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51aeb065f7df44ce9bae3db054130669'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:04.557322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1230/2474] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:04 2022] DELETE /identity/v3/users/51aeb065f7df44ce9bae3db054130669 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:04.936723 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:04.936723 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:04.951859 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530b72a-02c5-44f4-8061-d65cf1970418 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:05.040637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a135af-1985-465a-b020-e2d11b7d94bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:05.041684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a135af-1985-465a-b020-e2d11b7d94bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:05.047670 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a135af-1985-465a-b020-e2d11b7d94bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:05.052051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a135af-1985-465a-b020-e2d11b7d94bd admin admin] PATH_INFO: `/v3/projects/d0ff4a58b25e4d3492c25c1251c1d3f3` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:05.055493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a135af-1985-465a-b020-e2d11b7d94bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ff4a58b25e4d3492c25c1251c1d3f3)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:05.057822 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a135af-1985-465a-b020-e2d11b7d94bd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:05.136341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f4a135af-1985-465a-b020-e2d11b7d94bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ff4a58b25e4d3492c25c1251c1d3f3', 'request_id': 'req-f4a135af-1985-465a-b020-e2d11b7d94bd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:05.137861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:05.146657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f4a135af-1985-465a-b020-e2d11b7d94bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ff4a58b25e4d3492c25c1251c1d3f3', 'request_id': 'req-f4a135af-1985-465a-b020-e2d11b7d94bd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:05.157172 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f4a135af-1985-465a-b020-e2d11b7d94bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ff4a58b25e4d3492c25c1251c1d3f3', 'request_id': 'req-f4a135af-1985-465a-b020-e2d11b7d94bd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:05.174280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f4a135af-1985-465a-b020-e2d11b7d94bd admin admin] The token cache is being invalidate because project d0ff4a58b25e4d3492c25c1251c1d3f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:05.175292 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f4a135af-1985-465a-b020-e2d11b7d94bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:05.176601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f4a135af-1985-465a-b020-e2d11b7d94bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:05.179740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1245/2475] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:04 2022] DELETE /identity/v3/projects/d0ff4a58b25e4d3492c25c1251c1d3f3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:05.185249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25b9a1-191d-4cb8-b6f9-84c7d556d188 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:05.185860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25b9a1-191d-4cb8-b6f9-84c7d556d188 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:05.186411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25b9a1-191d-4cb8-b6f9-84c7d556d188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:05.199512 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:05.218555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-ef25b9a1-191d-4cb8-b6f9-84c7d556d188 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:05.221840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ef25b9a1-191d-4cb8-b6f9-84c7d556d188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:05.300460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1231/2476] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:34:05.309164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee75e16-16a5-448c-9d6e-03b8c78bd16d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:05.309456 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee75e16-16a5-448c-9d6e-03b8c78bd16d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:05.309755 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee75e16-16a5-448c-9d6e-03b8c78bd16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:05.331208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2ee75e16-16a5-448c-9d6e-03b8c78bd16d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:05.332507 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2ee75e16-16a5-448c-9d6e-03b8c78bd16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:05.344993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1246/2477] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:05.345967 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25b9a1-191d-4cb8-b6f9-84c7d556d188 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:05.358233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25b9a1-191d-4cb8-b6f9-84c7d556d188 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:05.364790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d3583-74b0-401d-bbd7-847a6acb9622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:05.365662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d3583-74b0-401d-bbd7-847a6acb9622 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:05.366653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d3583-74b0-401d-bbd7-847a6acb9622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:05.367102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d3583-74b0-401d-bbd7-847a6acb9622 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:05.367948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d3583-74b0-401d-bbd7-847a6acb9622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:34:05.368437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d3583-74b0-401d-bbd7-847a6acb9622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:05.370229 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d3583-74b0-401d-bbd7-847a6acb9622 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:05.392263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1232/2478] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:34:05.393487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee75e16-16a5-448c-9d6e-03b8c78bd16d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:05.411277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee75e16-16a5-448c-9d6e-03b8c78bd16d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:05.424697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9cf44-004e-47a9-8e3a-039ffb34fad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:05.425330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9cf44-004e-47a9-8e3a-039ffb34fad2 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:05.425668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9cf44-004e-47a9-8e3a-039ffb34fad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:05.425950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9cf44-004e-47a9-8e3a-039ffb34fad2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:05.426608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9cf44-004e-47a9-8e3a-039ffb34fad2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:05.428639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9cf44-004e-47a9-8e3a-039ffb34fad2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:05.458106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1247/2479] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:05 2022] POST /identity/v3/projects => generated 393 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:05.463446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d3583-74b0-401d-bbd7-847a6acb9622 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:05.476134 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d3583-74b0-401d-bbd7-847a6acb9622 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:05.552604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b52111-1d8d-4ae9-a8b6-b87ff8845288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:05.553438 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b52111-1d8d-4ae9-a8b6-b87ff8845288 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:05.553821 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b52111-1d8d-4ae9-a8b6-b87ff8845288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:05.554701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b52111-1d8d-4ae9-a8b6-b87ff8845288 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:05.556232 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b52111-1d8d-4ae9-a8b6-b87ff8845288 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:05.558125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b52111-1d8d-4ae9-a8b6-b87ff8845288 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:05.596311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1233/2480] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:05 2022] POST /identity/v3/users => generated 297 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:05.598755 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9cf44-004e-47a9-8e3a-039ffb34fad2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:05.613717 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9cf44-004e-47a9-8e3a-039ffb34fad2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:05.618986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600b318-ca7a-411e-a4a3-7e8b4d9acec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:05.620303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600b318-ca7a-411e-a4a3-7e8b4d9acec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:05.620762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600b318-ca7a-411e-a4a3-7e8b4d9acec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:05.621213 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600b318-ca7a-411e-a4a3-7e8b4d9acec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:05.622111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600b318-ca7a-411e-a4a3-7e8b4d9acec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:05.624115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600b318-ca7a-411e-a4a3-7e8b4d9acec3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:05.637085 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1248/2481] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:05.641651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b52111-1d8d-4ae9-a8b6-b87ff8845288 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:05.653010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b52111-1d8d-4ae9-a8b6-b87ff8845288 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:05.667424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8e37e-424d-4106-91e6-5a524cab6f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:05.668228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8e37e-424d-4106-91e6-5a524cab6f65 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:05.668640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8e37e-424d-4106-91e6-5a524cab6f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:05.669013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8e37e-424d-4106-91e6-5a524cab6f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:05.669766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8e37e-424d-4106-91e6-5a524cab6f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:05.671558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8e37e-424d-4106-91e6-5a524cab6f65 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:05.690485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1234/2482] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:05.695383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600b318-ca7a-411e-a4a3-7e8b4d9acec3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:05.707617 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600b318-ca7a-411e-a4a3-7e8b4d9acec3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:05.714111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7deb7e-f116-4625-b8a9-e3925462d18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:05.714818 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7deb7e-f116-4625-b8a9-e3925462d18e admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:05.715108 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7deb7e-f116-4625-b8a9-e3925462d18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:05.715481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7deb7e-f116-4625-b8a9-e3925462d18e admin admin] PATH_INFO: `/v3/projects/1a3c36fd2246413d88822f81dc07162d/users/6c9e1d7d725a4bab8526c045bd95274f/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:05.735409 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7deb7e-f116-4625-b8a9-e3925462d18e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a3c36fd2246413d88822f81dc07162d, user_id=6c9e1d7d725a4bab8526c045bd95274f, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:05.738313 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7deb7e-f116-4625-b8a9-e3925462d18e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:05.753616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1249/2483] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:05 2022] PUT /identity/v3/projects/1a3c36fd2246413d88822f81dc07162d/users/6c9e1d7d725a4bab8526c045bd95274f/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:05.764663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab5550-3429-4c94-a19a-b8f2ea8c5e29 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:05.764944 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab5550-3429-4c94-a19a-b8f2ea8c5e29 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:05.765349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab5550-3429-4c94-a19a-b8f2ea8c5e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:05.798589 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-68ab5550-3429-4c94-a19a-b8f2ea8c5e29 None None] MFA Rules not processed for user `6c9e1d7d725a4bab8526c045bd95274f`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:05.800121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-68ab5550-3429-4c94-a19a-b8f2ea8c5e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:05.930142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:05.944212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1235/2484] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:05.949094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7deb7e-f116-4625-b8a9-e3925462d18e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:06.014966 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7deb7e-f116-4625-b8a9-e3925462d18e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:06.020454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51de7c-a21a-473d-a844-093b533d253d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:06.021323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51de7c-a21a-473d-a844-093b533d253d admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:06.021610 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51de7c-a21a-473d-a844-093b533d253d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:06.021888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51de7c-a21a-473d-a844-093b533d253d admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:06.022528 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51de7c-a21a-473d-a844-093b533d253d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:06.024506 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51de7c-a21a-473d-a844-093b533d253d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:06.053318 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1250/2485] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:05 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:06.057704 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab5550-3429-4c94-a19a-b8f2ea8c5e29 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:06.069456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab5550-3429-4c94-a19a-b8f2ea8c5e29 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:06.129820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43373c-0f38-42fa-a568-1698045e4607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:06.131046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43373c-0f38-42fa-a568-1698045e4607 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:06.131287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43373c-0f38-42fa-a568-1698045e4607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:06.131590 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43373c-0f38-42fa-a568-1698045e4607 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:06.132430 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43373c-0f38-42fa-a568-1698045e4607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:06.134525 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43373c-0f38-42fa-a568-1698045e4607 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:06.158963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1236/2486] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:06 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:34:06.164563 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51de7c-a21a-473d-a844-093b533d253d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:06.175629 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51de7c-a21a-473d-a844-093b533d253d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:06.181053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd9577-3761-4665-9203-595be3e833ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:06.181663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd9577-3761-4665-9203-595be3e833ac admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:06.181939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd9577-3761-4665-9203-595be3e833ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:06.182365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd9577-3761-4665-9203-595be3e833ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:06.183167 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd9577-3761-4665-9203-595be3e833ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:06.184998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd9577-3761-4665-9203-595be3e833ac admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:06.198684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1251/2487] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:06.202919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43373c-0f38-42fa-a568-1698045e4607 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:06.215934 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43373c-0f38-42fa-a568-1698045e4607 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:06.221423 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f770ee-a207-4a04-9e3a-ed592b3c41cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:06.222173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f770ee-a207-4a04-9e3a-ed592b3c41cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:06.222465 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f770ee-a207-4a04-9e3a-ed592b3c41cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:06.222724 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f770ee-a207-4a04-9e3a-ed592b3c41cf admin admin] PATH_INFO: `/v3/projects/f8b91853797d4eae80dd922c7b127c66/users/038389412d7b49c8b07c89a299051418/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:06.249466 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f770ee-a207-4a04-9e3a-ed592b3c41cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b91853797d4eae80dd922c7b127c66, user_id=038389412d7b49c8b07c89a299051418, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:06.252773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f770ee-a207-4a04-9e3a-ed592b3c41cf admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:06.272004 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1237/2488] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:06 2022] PUT /identity/v3/projects/f8b91853797d4eae80dd922c7b127c66/users/038389412d7b49c8b07c89a299051418/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:34:06.276609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd9577-3761-4665-9203-595be3e833ac admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:06.288544 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd9577-3761-4665-9203-595be3e833ac admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:06.364323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:06.389429 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9864ef7-4d9a-4a5b-8f44-f3bf074774f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:06.390051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9864ef7-4d9a-4a5b-8f44-f3bf074774f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:06.390336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9864ef7-4d9a-4a5b-8f44-f3bf074774f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:06.390616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9864ef7-4d9a-4a5b-8f44-f3bf074774f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:06.391265 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9864ef7-4d9a-4a5b-8f44-f3bf074774f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:06.392915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9864ef7-4d9a-4a5b-8f44-f3bf074774f1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:06.403791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1252/2489] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:06.408157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f770ee-a207-4a04-9e3a-ed592b3c41cf admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:06.421097 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f770ee-a207-4a04-9e3a-ed592b3c41cf admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:06.425961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c46c2-3fcb-471e-8852-cf21e5e29c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:06.426867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c46c2-3fcb-471e-8852-cf21e5e29c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:06.427745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c46c2-3fcb-471e-8852-cf21e5e29c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:06.428055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c46c2-3fcb-471e-8852-cf21e5e29c1c admin admin] PATH_INFO: `/v3/projects/f8b91853797d4eae80dd922c7b127c66/users/038389412d7b49c8b07c89a299051418/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:06.430096 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c46c2-3fcb-471e-8852-cf21e5e29c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b91853797d4eae80dd922c7b127c66, user_id=038389412d7b49c8b07c89a299051418, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:06.432802 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c46c2-3fcb-471e-8852-cf21e5e29c1c admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:06.445730 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1238/2490] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:06 2022] PUT /identity/v3/projects/f8b91853797d4eae80dd922c7b127c66/users/038389412d7b49c8b07c89a299051418/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:06.519037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7d72e-0542-47e5-9ed4-5e66ca77b712 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:06.519326 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7d72e-0542-47e5-9ed4-5e66ca77b712 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:06.519644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7d72e-0542-47e5-9ed4-5e66ca77b712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:06.549691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c3a7d72e-0542-47e5-9ed4-5e66ca77b712 None None] MFA Rules not processed for user `038389412d7b49c8b07c89a299051418`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:06.550935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c3a7d72e-0542-47e5-9ed4-5e66ca77b712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:06.624013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1253/2491] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:06 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:06.635140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c46c2-3fcb-471e-8852-cf21e5e29c1c admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:06.636847 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-0b2c46c2-3fcb-471e-8852-cf21e5e29c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:06.696495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c46c2-3fcb-471e-8852-cf21e5e29c1c admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:06.700294 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca131c8-7809-473c-9bdb-17445102d0fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:06.701067 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca131c8-7809-473c-9bdb-17445102d0fe service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:06.701166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca131c8-7809-473c-9bdb-17445102d0fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:06.701435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca131c8-7809-473c-9bdb-17445102d0fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:06.710905 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca131c8-7809-473c-9bdb-17445102d0fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:06.712378 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca131c8-7809-473c-9bdb-17445102d0fe service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:06.721950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1239/2492] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:06 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:06.781613 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7d72e-0542-47e5-9ed4-5e66ca77b712 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:06.783187 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c3a7d72e-0542-47e5-9ed4-5e66ca77b712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:06.795120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7d72e-0542-47e5-9ed4-5e66ca77b712 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:06.798712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ed11d-c99e-4dd0-b00e-a986bfbcf406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:06.799303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ed11d-c99e-4dd0-b00e-a986bfbcf406 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:06.799596 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ed11d-c99e-4dd0-b00e-a986bfbcf406 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:06.799876 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ed11d-c99e-4dd0-b00e-a986bfbcf406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:06.860269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ed11d-c99e-4dd0-b00e-a986bfbcf406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:06.861716 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ed11d-c99e-4dd0-b00e-a986bfbcf406 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:06.871150 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1254/2493] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:06 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:10.299853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca131c8-7809-473c-9bdb-17445102d0fe service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:10.357286 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:10.365605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca131c8-7809-473c-9bdb-17445102d0fe service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:10.370745 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8cfe2-c12e-4bc3-b5c9-bd641a41459e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:10.371367 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8cfe2-c12e-4bc3-b5c9-bd641a41459e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:10.371744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8cfe2-c12e-4bc3-b5c9-bd641a41459e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:10.372018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8cfe2-c12e-4bc3-b5c9-bd641a41459e admin admin] PATH_INFO: `/v3/users/6c9e1d7d725a4bab8526c045bd95274f` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:10.373115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8cfe2-c12e-4bc3-b5c9-bd641a41459e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c9e1d7d725a4bab8526c045bd95274f)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:10.375044 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8cfe2-c12e-4bc3-b5c9-bd641a41459e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:10.423458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-99c8cfe2-c12e-4bc3-b5c9-bd641a41459e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c9e1d7d725a4bab8526c045bd95274f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:10.435923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-99c8cfe2-c12e-4bc3-b5c9-bd641a41459e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c9e1d7d725a4bab8526c045bd95274f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:10.443352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-99c8cfe2-c12e-4bc3-b5c9-bd641a41459e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c9e1d7d725a4bab8526c045bd95274f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:10.444000 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-99c8cfe2-c12e-4bc3-b5c9-bd641a41459e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c9e1d7d725a4bab8526c045bd95274f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:10.450935 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-99c8cfe2-c12e-4bc3-b5c9-bd641a41459e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c9e1d7d725a4bab8526c045bd95274f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:10.456815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1240/2494] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:10 2022] DELETE /identity/v3/users/6c9e1d7d725a4bab8526c045bd95274f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:10.461305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be239a79-b7b9-4f7d-8f27-600464d2a803 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:10.461855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be239a79-b7b9-4f7d-8f27-600464d2a803 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:10.461964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be239a79-b7b9-4f7d-8f27-600464d2a803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:10.480473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-be239a79-b7b9-4f7d-8f27-600464d2a803 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:10.481667 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-be239a79-b7b9-4f7d-8f27-600464d2a803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:10.539628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1255/2495] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:10.549503 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8cfe2-c12e-4bc3-b5c9-bd641a41459e admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:10.550930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-99c8cfe2-c12e-4bc3-b5c9-bd641a41459e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:10.609740 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8cfe2-c12e-4bc3-b5c9-bd641a41459e admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:10.615816 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afa222-ea8c-4238-a0a4-8d62698fc228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:10.615990 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afa222-ea8c-4238-a0a4-8d62698fc228 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:10.616415 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afa222-ea8c-4238-a0a4-8d62698fc228 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:10.616806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afa222-ea8c-4238-a0a4-8d62698fc228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:10.628931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6afa222-ea8c-4238-a0a4-8d62698fc228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:10.631149 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6afa222-ea8c-4238-a0a4-8d62698fc228 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:10.641644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1241/2496] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:10.992392 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be239a79-b7b9-4f7d-8f27-600464d2a803 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:10.996475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-be239a79-b7b9-4f7d-8f27-600464d2a803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:11.019114 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be239a79-b7b9-4f7d-8f27-600464d2a803 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:11.025515 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:11.026446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:11.026829 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:11.027202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] PATH_INFO: `/v3/projects/1a3c36fd2246413d88822f81dc07162d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:11.028763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a3c36fd2246413d88822f81dc07162d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:11.041278 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:11.055527 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:11.140310 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a3c36fd2246413d88822f81dc07162d', 'request_id': 'req-bdc6fde3-6252-4f48-a235-a440f41e17c9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.149734 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a3c36fd2246413d88822f81dc07162d', 'request_id': 'req-bdc6fde3-6252-4f48-a235-a440f41e17c9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.159791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a3c36fd2246413d88822f81dc07162d', 'request_id': 'req-bdc6fde3-6252-4f48-a235-a440f41e17c9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.178422 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] The token cache is being invalidate because project 1a3c36fd2246413d88822f81dc07162d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:11.178896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.179703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.181524 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1256/2497] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:10 2022] DELETE /identity/v3/projects/1a3c36fd2246413d88822f81dc07162d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:34:11.185757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afa222-ea8c-4238-a0a4-8d62698fc228 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:11.196208 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afa222-ea8c-4238-a0a4-8d62698fc228 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:11.268652 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed1167-1437-494f-b112-bfb4317883b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:11.272262 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed1167-1437-494f-b112-bfb4317883b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:11.272545 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed1167-1437-494f-b112-bfb4317883b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:11.272822 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed1167-1437-494f-b112-bfb4317883b0 admin admin] PATH_INFO: `/v3/users/038389412d7b49c8b07c89a299051418` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:11.274023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed1167-1437-494f-b112-bfb4317883b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=038389412d7b49c8b07c89a299051418)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:11.276547 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed1167-1437-494f-b112-bfb4317883b0 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:11.316442 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:11.358565 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-b8ed1167-1437-494f-b112-bfb4317883b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '038389412d7b49c8b07c89a299051418'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.373624 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-b8ed1167-1437-494f-b112-bfb4317883b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '038389412d7b49c8b07c89a299051418'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.385951 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-b8ed1167-1437-494f-b112-bfb4317883b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '038389412d7b49c8b07c89a299051418'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.387054 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-b8ed1167-1437-494f-b112-bfb4317883b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '038389412d7b49c8b07c89a299051418'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.399548 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-b8ed1167-1437-494f-b112-bfb4317883b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '038389412d7b49c8b07c89a299051418'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.407116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1242/2498] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:11 2022] DELETE /identity/v3/users/038389412d7b49c8b07c89a299051418 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:11.672261 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:11.673878 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:11.687648 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6fde3-6252-4f48-a235-a440f41e17c9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:11.747466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663a7c2-fd4f-4799-9354-9c8bb981f319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:11.748183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663a7c2-fd4f-4799-9354-9c8bb981f319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:11.748466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663a7c2-fd4f-4799-9354-9c8bb981f319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:11.748738 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663a7c2-fd4f-4799-9354-9c8bb981f319 admin admin] PATH_INFO: `/v3/projects/f8b91853797d4eae80dd922c7b127c66` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:11.749935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663a7c2-fd4f-4799-9354-9c8bb981f319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8b91853797d4eae80dd922c7b127c66)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:11.751848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663a7c2-fd4f-4799-9354-9c8bb981f319 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:11.798206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c663a7c2-fd4f-4799-9354-9c8bb981f319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b91853797d4eae80dd922c7b127c66', 'request_id': 'req-c663a7c2-fd4f-4799-9354-9c8bb981f319'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.803785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c663a7c2-fd4f-4799-9354-9c8bb981f319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b91853797d4eae80dd922c7b127c66', 'request_id': 'req-c663a7c2-fd4f-4799-9354-9c8bb981f319'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.811272 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c663a7c2-fd4f-4799-9354-9c8bb981f319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b91853797d4eae80dd922c7b127c66', 'request_id': 'req-c663a7c2-fd4f-4799-9354-9c8bb981f319'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.823507 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c663a7c2-fd4f-4799-9354-9c8bb981f319 admin admin] The token cache is being invalidate because project f8b91853797d4eae80dd922c7b127c66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:11.823817 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c663a7c2-fd4f-4799-9354-9c8bb981f319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.824450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c663a7c2-fd4f-4799-9354-9c8bb981f319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:11.825930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1257/2499] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:11 2022] DELETE /identity/v3/projects/f8b91853797d4eae80dd922c7b127c66 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:11.830808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3f7c5-8eae-4eb7-bb39-2a23506cce8c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:11.831163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3f7c5-8eae-4eb7-bb39-2a23506cce8c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:11.831516 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3f7c5-8eae-4eb7-bb39-2a23506cce8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:11.852081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-5dc3f7c5-8eae-4eb7-bb39-2a23506cce8c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:11.853424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5dc3f7c5-8eae-4eb7-bb39-2a23506cce8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:11.915413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1243/2500] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:11.922823 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46109e-2ac4-4bf3-a291-554da9448cac None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:11.923108 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46109e-2ac4-4bf3-a291-554da9448cac None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:11.923382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46109e-2ac4-4bf3-a291-554da9448cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:11.943831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3b46109e-2ac4-4bf3-a291-554da9448cac None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:11.945013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3b46109e-2ac4-4bf3-a291-554da9448cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:11.952791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1258/2501] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:34:11.956826 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3f7c5-8eae-4eb7-bb39-2a23506cce8c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:11.966483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3f7c5-8eae-4eb7-bb39-2a23506cce8c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:11.971089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2631c4-7017-4271-9b42-c98ee92258f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:11.971736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2631c4-7017-4271-9b42-c98ee92258f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:11.972021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2631c4-7017-4271-9b42-c98ee92258f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:11.972296 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2631c4-7017-4271-9b42-c98ee92258f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:11.972935 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2631c4-7017-4271-9b42-c98ee92258f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:34:11.973239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2631c4-7017-4271-9b42-c98ee92258f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:11.974911 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2631c4-7017-4271-9b42-c98ee92258f0 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:11.986626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1244/2502] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:34:11.991399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46109e-2ac4-4bf3-a291-554da9448cac None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:12.000550 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46109e-2ac4-4bf3-a291-554da9448cac None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:12.005840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae23c9-21c7-4917-b89c-458499bf30bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:12.006440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae23c9-21c7-4917-b89c-458499bf30bf admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:12.006706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae23c9-21c7-4917-b89c-458499bf30bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:12.006973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae23c9-21c7-4917-b89c-458499bf30bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:12.007628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae23c9-21c7-4917-b89c-458499bf30bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:12.009557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae23c9-21c7-4917-b89c-458499bf30bf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:12.025714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:12.044945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1259/2503] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:11 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:12.048627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2631c4-7017-4271-9b42-c98ee92258f0 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:12.059484 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2631c4-7017-4271-9b42-c98ee92258f0 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:12.117046 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658afad7-0bca-4fa0-b037-4d490a9563f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:12.117825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658afad7-0bca-4fa0-b037-4d490a9563f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:12.118177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658afad7-0bca-4fa0-b037-4d490a9563f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:12.118551 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658afad7-0bca-4fa0-b037-4d490a9563f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:12.119354 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658afad7-0bca-4fa0-b037-4d490a9563f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:12.121710 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658afad7-0bca-4fa0-b037-4d490a9563f5 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:12.146550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1245/2504] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:12 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:12.151136 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae23c9-21c7-4917-b89c-458499bf30bf admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:12.161711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae23c9-21c7-4917-b89c-458499bf30bf admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:12.166776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9d53e-6cdb-42a3-9065-f4054bc638b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:12.167405 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9d53e-6cdb-42a3-9065-f4054bc638b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:12.167732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9d53e-6cdb-42a3-9065-f4054bc638b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:12.168013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9d53e-6cdb-42a3-9065-f4054bc638b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:12.168698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9d53e-6cdb-42a3-9065-f4054bc638b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:12.170249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9d53e-6cdb-42a3-9065-f4054bc638b0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:12.183685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1260/2505] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:12.185213 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658afad7-0bca-4fa0-b037-4d490a9563f5 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:12.195943 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658afad7-0bca-4fa0-b037-4d490a9563f5 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:12.200920 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fd57e-6894-4af6-9b29-ffb04db642b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:12.201622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fd57e-6894-4af6-9b29-ffb04db642b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:12.201829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fd57e-6894-4af6-9b29-ffb04db642b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:12.202077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fd57e-6894-4af6-9b29-ffb04db642b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:12.202730 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fd57e-6894-4af6-9b29-ffb04db642b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:12.204360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fd57e-6894-4af6-9b29-ffb04db642b9 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:12.216464 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1246/2506] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:12.220681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9d53e-6cdb-42a3-9065-f4054bc638b0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:12.230592 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9d53e-6cdb-42a3-9065-f4054bc638b0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:12.235258 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9722a3-b486-4295-ac68-3d17a31b28e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:12.235928 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9722a3-b486-4295-ac68-3d17a31b28e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:12.236215 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9722a3-b486-4295-ac68-3d17a31b28e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:12.236493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9722a3-b486-4295-ac68-3d17a31b28e9 admin admin] PATH_INFO: `/v3/projects/8655dc481389478a9e3a78929dbf11aa/users/715d0dc80f204a0fb1f8db5da40003ca/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:12.252145 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9722a3-b486-4295-ac68-3d17a31b28e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8655dc481389478a9e3a78929dbf11aa, user_id=715d0dc80f204a0fb1f8db5da40003ca, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:12.254716 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9722a3-b486-4295-ac68-3d17a31b28e9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:12.268914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1261/2507] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:12 2022] PUT /identity/v3/projects/8655dc481389478a9e3a78929dbf11aa/users/715d0dc80f204a0fb1f8db5da40003ca/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:12.273800 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8a593-8f5e-4239-8df2-82899d1487d8 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:12.274089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8a593-8f5e-4239-8df2-82899d1487d8 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:12.274365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8a593-8f5e-4239-8df2-82899d1487d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:12.282064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:12.296728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-3bf8a593-8f5e-4239-8df2-82899d1487d8 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:12.297958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3bf8a593-8f5e-4239-8df2-82899d1487d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:12.373115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1247/2508] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:12.384098 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9722a3-b486-4295-ac68-3d17a31b28e9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:12.386008 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bc9722a3-b486-4295-ac68-3d17a31b28e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:12.449356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9722a3-b486-4295-ac68-3d17a31b28e9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:12.453066 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15c6e0-6aae-45d6-b0f2-3b73509333e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:12.453672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15c6e0-6aae-45d6-b0f2-3b73509333e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:12.453950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15c6e0-6aae-45d6-b0f2-3b73509333e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:12.454228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15c6e0-6aae-45d6-b0f2-3b73509333e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:12.464196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15c6e0-6aae-45d6-b0f2-3b73509333e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:12.465762 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15c6e0-6aae-45d6-b0f2-3b73509333e4 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:12.476149 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1262/2509] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:13.266399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cabe11-17a9-4635-b28e-b6d633ac7f54 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:13.266877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cabe11-17a9-4635-b28e-b6d633ac7f54 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:13.267478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cabe11-17a9-4635-b28e-b6d633ac7f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:13.310708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-92cabe11-17a9-4635-b28e-b6d633ac7f54 None None] MFA Rules not processed for user `715d0dc80f204a0fb1f8db5da40003ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:13.312110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-92cabe11-17a9-4635-b28e-b6d633ac7f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:13.387197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15c6e0-6aae-45d6-b0f2-3b73509333e4 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:13.388660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5b15c6e0-6aae-45d6-b0f2-3b73509333e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:13.415162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15c6e0-6aae-45d6-b0f2-3b73509333e4 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:13.427178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159eefb-84a0-48c9-9f20-0bce9604ce46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:13.428432 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159eefb-84a0-48c9-9f20-0bce9604ce46 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:13.428722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159eefb-84a0-48c9-9f20-0bce9604ce46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:13.429001 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159eefb-84a0-48c9-9f20-0bce9604ce46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:13.435597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f159eefb-84a0-48c9-9f20-0bce9604ce46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:13.467082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159eefb-84a0-48c9-9f20-0bce9604ce46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:13.467555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159eefb-84a0-48c9-9f20-0bce9604ce46 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:13.487274 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1248/2510] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:13.487816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1263/2511] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:34:13.488411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cabe11-17a9-4635-b28e-b6d633ac7f54 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:13.502195 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cabe11-17a9-4635-b28e-b6d633ac7f54 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:13.510137 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea023f6-d5d0-4dcd-814d-ed78b79783b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:13.510770 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea023f6-d5d0-4dcd-814d-ed78b79783b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:13.511056 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea023f6-d5d0-4dcd-814d-ed78b79783b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:13.511331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea023f6-d5d0-4dcd-814d-ed78b79783b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:13.512008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea023f6-d5d0-4dcd-814d-ed78b79783b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:13.514078 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea023f6-d5d0-4dcd-814d-ed78b79783b7 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:13.545917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1249/2512] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:13 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:13.550552 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159eefb-84a0-48c9-9f20-0bce9604ce46 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:13.560659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159eefb-84a0-48c9-9f20-0bce9604ce46 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:13.626404 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbd21d-c208-4d84-ba3e-2c04733544a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:13.627208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbd21d-c208-4d84-ba3e-2c04733544a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:13.627685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbd21d-c208-4d84-ba3e-2c04733544a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:13.628091 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbd21d-c208-4d84-ba3e-2c04733544a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:13.628861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbd21d-c208-4d84-ba3e-2c04733544a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:13.631665 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbd21d-c208-4d84-ba3e-2c04733544a0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:13.658420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1264/2513] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:13 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:34:13.659625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea023f6-d5d0-4dcd-814d-ed78b79783b7 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:13.672804 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea023f6-d5d0-4dcd-814d-ed78b79783b7 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:13.680857 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad5a04-f79e-4512-a8db-1b0398d423a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:13.681555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad5a04-f79e-4512-a8db-1b0398d423a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:13.682395 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad5a04-f79e-4512-a8db-1b0398d423a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:13.682858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad5a04-f79e-4512-a8db-1b0398d423a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:13.683618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad5a04-f79e-4512-a8db-1b0398d423a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:13.685515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad5a04-f79e-4512-a8db-1b0398d423a4 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:13.699937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1250/2514] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:34:13.705257 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbd21d-c208-4d84-ba3e-2c04733544a0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:13.715370 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbd21d-c208-4d84-ba3e-2c04733544a0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:13.720573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d746b-0ac0-4be8-b404-9a1c23f14a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:13.721227 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d746b-0ac0-4be8-b404-9a1c23f14a85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:13.721576 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d746b-0ac0-4be8-b404-9a1c23f14a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:13.721852 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d746b-0ac0-4be8-b404-9a1c23f14a85 admin admin] PATH_INFO: `/v3/projects/2e5c38695132459eb597cef9cbc5c8b6/users/8de67995f78840eb94afe9fe4d86e3f8/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:13.737852 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d746b-0ac0-4be8-b404-9a1c23f14a85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e5c38695132459eb597cef9cbc5c8b6, user_id=8de67995f78840eb94afe9fe4d86e3f8, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:13.741142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d746b-0ac0-4be8-b404-9a1c23f14a85 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:13.756413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1265/2515] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:13 2022] PUT /identity/v3/projects/2e5c38695132459eb597cef9cbc5c8b6/users/8de67995f78840eb94afe9fe4d86e3f8/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:13.760247 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad5a04-f79e-4512-a8db-1b0398d423a4 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:13.770244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad5a04-f79e-4512-a8db-1b0398d423a4 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:13.829099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bc123-d7f3-4b20-b985-4b72f2ac0b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:13.829723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bc123-d7f3-4b20-b985-4b72f2ac0b59 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:13.830004 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bc123-d7f3-4b20-b985-4b72f2ac0b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:13.830279 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bc123-d7f3-4b20-b985-4b72f2ac0b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:13.830924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bc123-d7f3-4b20-b985-4b72f2ac0b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:13.832569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bc123-d7f3-4b20-b985-4b72f2ac0b59 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:13.834112 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:13.847920 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1251/2516] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:34:13.849746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d746b-0ac0-4be8-b404-9a1c23f14a85 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:13.860107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d746b-0ac0-4be8-b404-9a1c23f14a85 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:13.865319 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af6162-0bf9-4b05-8e9f-4101b41fec22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:13.865963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af6162-0bf9-4b05-8e9f-4101b41fec22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:13.866239 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af6162-0bf9-4b05-8e9f-4101b41fec22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:13.866506 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af6162-0bf9-4b05-8e9f-4101b41fec22 admin admin] PATH_INFO: `/v3/projects/2e5c38695132459eb597cef9cbc5c8b6/users/8de67995f78840eb94afe9fe4d86e3f8/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:13.868446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af6162-0bf9-4b05-8e9f-4101b41fec22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e5c38695132459eb597cef9cbc5c8b6, user_id=8de67995f78840eb94afe9fe4d86e3f8, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:13.871067 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af6162-0bf9-4b05-8e9f-4101b41fec22 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:13.877900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:13.884994 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1266/2517] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:13 2022] PUT /identity/v3/projects/2e5c38695132459eb597cef9cbc5c8b6/users/8de67995f78840eb94afe9fe4d86e3f8/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:14.554071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31df5d0-b12b-4bf3-b76e-44ef2e988000 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:14.554555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31df5d0-b12b-4bf3-b76e-44ef2e988000 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:14.554638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31df5d0-b12b-4bf3-b76e-44ef2e988000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:14.571880 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7e0cf-1e35-49a0-9503-f41fd51475a4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:14.572765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7e0cf-1e35-49a0-9503-f41fd51475a4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:14.573257 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7e0cf-1e35-49a0-9503-f41fd51475a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:14.592217 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b31df5d0-b12b-4bf3-b76e-44ef2e988000 None None] MFA Rules not processed for user `8de67995f78840eb94afe9fe4d86e3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:14.593858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b31df5d0-b12b-4bf3-b76e-44ef2e988000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:14.605870 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-36c7e0cf-1e35-49a0-9503-f41fd51475a4 None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:14.607946 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-36c7e0cf-1e35-49a0-9503-f41fd51475a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:14.694757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1252/2518] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:14.712844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31df5d0-b12b-4bf3-b76e-44ef2e988000 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:14.715148 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b31df5d0-b12b-4bf3-b76e-44ef2e988000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:14.742386 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1267/2519] 10.222.0.27 () {58 vars in 967 bytes} [Mon Sep 12 23:34:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:34:14.813208 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7e0cf-1e35-49a0-9503-f41fd51475a4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:14.815028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-36c7e0cf-1e35-49a0-9503-f41fd51475a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:14.816505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31df5d0-b12b-4bf3-b76e-44ef2e988000 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:14.822893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30339c-dc7c-4ebf-8b2a-8dd8e70fe99c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:14.824588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30339c-dc7c-4ebf-8b2a-8dd8e70fe99c service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:14.825066 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30339c-dc7c-4ebf-8b2a-8dd8e70fe99c service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:14.826160 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30339c-dc7c-4ebf-8b2a-8dd8e70fe99c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:14.929802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7e0cf-1e35-49a0-9503-f41fd51475a4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:14.936754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88e353-3db1-49a2-840d-30ce973b4ad4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:14.937525 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88e353-3db1-49a2-840d-30ce973b4ad4 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:14.937810 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88e353-3db1-49a2-840d-30ce973b4ad4 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:14.938082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88e353-3db1-49a2-840d-30ce973b4ad4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:14.944809 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30339c-dc7c-4ebf-8b2a-8dd8e70fe99c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:14.946307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30339c-dc7c-4ebf-8b2a-8dd8e70fe99c service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:14.949397 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88e353-3db1-49a2-840d-30ce973b4ad4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:14.951896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88e353-3db1-49a2-840d-30ce973b4ad4 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:14.955470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1253/2520] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:34:14.964793 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1268/2521] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:34:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 23:34:16.435699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30339c-dc7c-4ebf-8b2a-8dd8e70fe99c service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:16.438339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-0c30339c-dc7c-4ebf-8b2a-8dd8e70fe99c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:16.454903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30339c-dc7c-4ebf-8b2a-8dd8e70fe99c service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:16.458539 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066c1d3-d2ef-4fc1-8d18-7a7791c9bbd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:16.459138 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066c1d3-d2ef-4fc1-8d18-7a7791c9bbd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:16.459421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066c1d3-d2ef-4fc1-8d18-7a7791c9bbd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:16.459852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066c1d3-d2ef-4fc1-8d18-7a7791c9bbd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:16.469401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066c1d3-d2ef-4fc1-8d18-7a7791c9bbd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:16.470841 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066c1d3-d2ef-4fc1-8d18-7a7791c9bbd6 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:16.480472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1254/2522] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:19.139924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88e353-3db1-49a2-840d-30ce973b4ad4 service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:19.141445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4e88e353-3db1-49a2-840d-30ce973b4ad4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:19.206534 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88e353-3db1-49a2-840d-30ce973b4ad4 service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:19.211090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23755d7f-8ba9-42ab-b023-b25dd4433c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:19.211748 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23755d7f-8ba9-42ab-b023-b25dd4433c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:19.212026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23755d7f-8ba9-42ab-b023-b25dd4433c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:19.212299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23755d7f-8ba9-42ab-b023-b25dd4433c1e admin admin] PATH_INFO: `/v3/users/7f56dde21c754cfe8253ec5c1b7b90d7` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:19.213411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23755d7f-8ba9-42ab-b023-b25dd4433c1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f56dde21c754cfe8253ec5c1b7b90d7)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:19.215308 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23755d7f-8ba9-42ab-b023-b25dd4433c1e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:19.254413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:19.263728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-23755d7f-8ba9-42ab-b023-b25dd4433c1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f56dde21c754cfe8253ec5c1b7b90d7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:19.270675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-23755d7f-8ba9-42ab-b023-b25dd4433c1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f56dde21c754cfe8253ec5c1b7b90d7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:19.282815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-23755d7f-8ba9-42ab-b023-b25dd4433c1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f56dde21c754cfe8253ec5c1b7b90d7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:19.290341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-23755d7f-8ba9-42ab-b023-b25dd4433c1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f56dde21c754cfe8253ec5c1b7b90d7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:19.290959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-23755d7f-8ba9-42ab-b023-b25dd4433c1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f56dde21c754cfe8253ec5c1b7b90d7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:19.297636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1269/2523] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:19 2022] DELETE /identity/v3/users/7f56dde21c754cfe8253ec5c1b7b90d7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:19.478991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066c1d3-d2ef-4fc1-8d18-7a7791c9bbd6 service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:19.490086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066c1d3-d2ef-4fc1-8d18-7a7791c9bbd6 service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:19.516292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:19.563759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421fa93-2a62-42cb-880b-c2b5661e5f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:19.564546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421fa93-2a62-42cb-880b-c2b5661e5f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:19.564961 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421fa93-2a62-42cb-880b-c2b5661e5f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:19.565344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421fa93-2a62-42cb-880b-c2b5661e5f3e admin admin] PATH_INFO: `/v3/projects/b8ffe803ce4b4d0181be62e1dfb094bf` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:19.566720 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421fa93-2a62-42cb-880b-c2b5661e5f3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8ffe803ce4b4d0181be62e1dfb094bf)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:19.568780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421fa93-2a62-42cb-880b-c2b5661e5f3e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:19.637225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5421fa93-2a62-42cb-880b-c2b5661e5f3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8ffe803ce4b4d0181be62e1dfb094bf', 'request_id': 'req-5421fa93-2a62-42cb-880b-c2b5661e5f3e'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:19.644038 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5421fa93-2a62-42cb-880b-c2b5661e5f3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8ffe803ce4b4d0181be62e1dfb094bf', 'request_id': 'req-5421fa93-2a62-42cb-880b-c2b5661e5f3e'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:19.653322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5421fa93-2a62-42cb-880b-c2b5661e5f3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8ffe803ce4b4d0181be62e1dfb094bf', 'request_id': 'req-5421fa93-2a62-42cb-880b-c2b5661e5f3e'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:19.665349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5421fa93-2a62-42cb-880b-c2b5661e5f3e admin admin] The token cache is being invalidate because project b8ffe803ce4b4d0181be62e1dfb094bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:19.665640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5421fa93-2a62-42cb-880b-c2b5661e5f3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:19.666289 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5421fa93-2a62-42cb-880b-c2b5661e5f3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:19.667815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1255/2524] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:19 2022] DELETE /identity/v3/projects/b8ffe803ce4b4d0181be62e1dfb094bf => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:19.673633 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969287bc-3122-4331-9286-b796783a2ea4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:19.673913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969287bc-3122-4331-9286-b796783a2ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:19.674194 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969287bc-3122-4331-9286-b796783a2ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:19.698427 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-969287bc-3122-4331-9286-b796783a2ea4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:19.699681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-969287bc-3122-4331-9286-b796783a2ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:19.762781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1270/2525] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:19.770829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70777ca8-e71c-4dd3-b3bc-d668c3f54b92 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:19.771121 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70777ca8-e71c-4dd3-b3bc-d668c3f54b92 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:19.771408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70777ca8-e71c-4dd3-b3bc-d668c3f54b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:19.791748 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-70777ca8-e71c-4dd3-b3bc-d668c3f54b92 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:19.792963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-70777ca8-e71c-4dd3-b3bc-d668c3f54b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:19.800829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1256/2526] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:19.805504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969287bc-3122-4331-9286-b796783a2ea4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:19.815051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969287bc-3122-4331-9286-b796783a2ea4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:19.819680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8b776-cef2-44d5-9f72-d54fcaa20e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:19.820269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8b776-cef2-44d5-9f72-d54fcaa20e7b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:19.820548 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8b776-cef2-44d5-9f72-d54fcaa20e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:19.820823 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8b776-cef2-44d5-9f72-d54fcaa20e7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:19.821466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8b776-cef2-44d5-9f72-d54fcaa20e7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:34:19.821770 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8b776-cef2-44d5-9f72-d54fcaa20e7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:19.823361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8b776-cef2-44d5-9f72-d54fcaa20e7b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:19.834602 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1271/2527] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:34:19.838699 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70777ca8-e71c-4dd3-b3bc-d668c3f54b92 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:19.847934 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70777ca8-e71c-4dd3-b3bc-d668c3f54b92 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:19.853093 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400614d3-e8e0-4464-b791-54b5bf877935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:19.853685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400614d3-e8e0-4464-b791-54b5bf877935 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:19.853966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400614d3-e8e0-4464-b791-54b5bf877935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:19.854239 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400614d3-e8e0-4464-b791-54b5bf877935 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:19.854872 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400614d3-e8e0-4464-b791-54b5bf877935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:19.856810 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400614d3-e8e0-4464-b791-54b5bf877935 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:19.882954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1257/2528] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:19 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:19.886981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8b776-cef2-44d5-9f72-d54fcaa20e7b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:19.896918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8b776-cef2-44d5-9f72-d54fcaa20e7b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:19.959919 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b7df8-d133-40bb-818e-267377f12d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:19.960656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b7df8-d133-40bb-818e-267377f12d38 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:19.960942 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b7df8-d133-40bb-818e-267377f12d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:19.961222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b7df8-d133-40bb-818e-267377f12d38 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:19.961859 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b7df8-d133-40bb-818e-267377f12d38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:19.963811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b7df8-d133-40bb-818e-267377f12d38 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:19.987453 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1272/2529] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:19 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:19.992246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400614d3-e8e0-4464-b791-54b5bf877935 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:20.002666 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400614d3-e8e0-4464-b791-54b5bf877935 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:20.007223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3d5ee-be5c-43fe-8d53-e433542599ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:20.007851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3d5ee-be5c-43fe-8d53-e433542599ba admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:20.008132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3d5ee-be5c-43fe-8d53-e433542599ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:20.008404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3d5ee-be5c-43fe-8d53-e433542599ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:20.009042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3d5ee-be5c-43fe-8d53-e433542599ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:20.010627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3d5ee-be5c-43fe-8d53-e433542599ba admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:20.020400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1258/2530] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:20.024506 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b7df8-d133-40bb-818e-267377f12d38 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:20.034764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b7df8-d133-40bb-818e-267377f12d38 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:20.040081 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c6125-904e-48eb-a6fc-125f85e25b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:20.040677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c6125-904e-48eb-a6fc-125f85e25b44 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:20.040957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c6125-904e-48eb-a6fc-125f85e25b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:20.041235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c6125-904e-48eb-a6fc-125f85e25b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:20.041874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c6125-904e-48eb-a6fc-125f85e25b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:20.043457 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c6125-904e-48eb-a6fc-125f85e25b44 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:20.054720 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1273/2531] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:20.058859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3d5ee-be5c-43fe-8d53-e433542599ba admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:20.068674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3d5ee-be5c-43fe-8d53-e433542599ba admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:20.073225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b0cdf-dadb-4d3e-8211-f6c829850671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:20.073858 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b0cdf-dadb-4d3e-8211-f6c829850671 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:20.074132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b0cdf-dadb-4d3e-8211-f6c829850671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:20.074402 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b0cdf-dadb-4d3e-8211-f6c829850671 admin admin] PATH_INFO: `/v3/projects/cecbca8dd4944b93836e910fa7f822ac/users/b97b92cc9451443ca102ee087b7b0a3b/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:20.088664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b0cdf-dadb-4d3e-8211-f6c829850671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cecbca8dd4944b93836e910fa7f822ac, user_id=b97b92cc9451443ca102ee087b7b0a3b, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:20.091288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b0cdf-dadb-4d3e-8211-f6c829850671 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:20.104499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1259/2532] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:20 2022] PUT /identity/v3/projects/cecbca8dd4944b93836e910fa7f822ac/users/b97b92cc9451443ca102ee087b7b0a3b/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:20.116830 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fa23a-e123-47d7-9d85-ed097926d946 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:20.117111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fa23a-e123-47d7-9d85-ed097926d946 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:20.117383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fa23a-e123-47d7-9d85-ed097926d946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:20.144672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-665fa23a-e123-47d7-9d85-ed097926d946 None None] MFA Rules not processed for user `b97b92cc9451443ca102ee087b7b0a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:20.146393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-665fa23a-e123-47d7-9d85-ed097926d946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:20.154768 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:20.215086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1274/2533] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:34:20.226579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b0cdf-dadb-4d3e-8211-f6c829850671 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:20.228403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8d5b0cdf-dadb-4d3e-8211-f6c829850671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:20.283452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:20.288617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b0cdf-dadb-4d3e-8211-f6c829850671 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:20.293541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fc6bd-ca1e-4d20-b1af-1311b187a73d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:20.294478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fc6bd-ca1e-4d20-b1af-1311b187a73d service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:20.295403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fc6bd-ca1e-4d20-b1af-1311b187a73d service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:20.295403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fc6bd-ca1e-4d20-b1af-1311b187a73d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:20.304064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46fc6bd-ca1e-4d20-b1af-1311b187a73d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:20.305709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46fc6bd-ca1e-4d20-b1af-1311b187a73d service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:20.315091 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1260/2534] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:21.090746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d9b85-c8e2-4b87-96c1-4d3a30843ad8 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:21.091036 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d9b85-c8e2-4b87-96c1-4d3a30843ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:21.091329 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d9b85-c8e2-4b87-96c1-4d3a30843ad8 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:21.092950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1275/2535] 10.222.0.27 () {54 vars in 921 bytes} [Mon Sep 12 23:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:34:21.099654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f484f-538c-4875-b516-c0d11992203f None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:21.100134 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f484f-538c-4875-b516-c0d11992203f None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:21.100273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f484f-538c-4875-b516-c0d11992203f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:21.122178 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c63f484f-538c-4875-b516-c0d11992203f None None] MFA Rules not processed for user `bc6e63e9fc3447d5a894a777a98fe773`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:21.123396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c63f484f-538c-4875-b516-c0d11992203f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:21.175018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1261/2536] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Sep 12 23:34:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:21.187129 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d9b85-c8e2-4b87-96c1-4d3a30843ad8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:21.188933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7d5d9b85-c8e2-4b87-96c1-4d3a30843ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:21.202297 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d9b85-c8e2-4b87-96c1-4d3a30843ad8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:21.207954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0067c-e064-4db5-b404-703f79b14e13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:21.208724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0067c-e064-4db5-b404-703f79b14e13 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:21.209120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0067c-e064-4db5-b404-703f79b14e13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:21.209398 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0067c-e064-4db5-b404-703f79b14e13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:21.216863 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0067c-e064-4db5-b404-703f79b14e13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:21.218277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0067c-e064-4db5-b404-703f79b14e13 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:21.226500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1276/2537] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:21.310912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bd3aa-13c5-498d-8a1b-e61491e01199 None None] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:21.311206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bd3aa-13c5-498d-8a1b-e61491e01199 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:21.311484 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bd3aa-13c5-498d-8a1b-e61491e01199 None None] PATH_INFO: `/` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:21.312874 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1262/2538] 10.222.0.27 () {54 vars in 921 bytes} [Mon Sep 12 23:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:34:21.318495 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b24db-af17-4aab-92d8-ffc15e540c70 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:21.318847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b24db-af17-4aab-92d8-ffc15e540c70 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:21.319122 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b24db-af17-4aab-92d8-ffc15e540c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:21.339042 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ee8b24db-af17-4aab-92d8-ffc15e540c70 None None] MFA Rules not processed for user `f5fa28ffdafd435cb2d93bd453add3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:21.340291 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ee8b24db-af17-4aab-92d8-ffc15e540c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:21.410725 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1277/2539] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Sep 12 23:34:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:21.421259 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bd3aa-13c5-498d-8a1b-e61491e01199 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:21.422942 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-048bd3aa-13c5-498d-8a1b-e61491e01199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:21.435975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bd3aa-13c5-498d-8a1b-e61491e01199 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:21.439509 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490ca5d-e042-48b6-962b-87d832b91915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:21.440157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490ca5d-e042-48b6-962b-87d832b91915 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:21.440440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490ca5d-e042-48b6-962b-87d832b91915 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:21.440717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490ca5d-e042-48b6-962b-87d832b91915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:21.451727 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490ca5d-e042-48b6-962b-87d832b91915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:21.453155 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490ca5d-e042-48b6-962b-87d832b91915 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:21.464446 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1263/2540] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:28.804169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b24db-af17-4aab-92d8-ffc15e540c70 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:28.864364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b24db-af17-4aab-92d8-ffc15e540c70 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:28.869819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3af2e-3572-4083-b59b-913da5c1e93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:28.870435 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3af2e-3572-4083-b59b-913da5c1e93e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:28.870714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3af2e-3572-4083-b59b-913da5c1e93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:28.870979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3af2e-3572-4083-b59b-913da5c1e93e admin admin] PATH_INFO: `/v3/users/b97b92cc9451443ca102ee087b7b0a3b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:28.872115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3af2e-3572-4083-b59b-913da5c1e93e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b97b92cc9451443ca102ee087b7b0a3b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:28.874015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3af2e-3572-4083-b59b-913da5c1e93e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:28.907459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:28.927086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-00a3af2e-3572-4083-b59b-913da5c1e93e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b97b92cc9451443ca102ee087b7b0a3b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:28.934757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-00a3af2e-3572-4083-b59b-913da5c1e93e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b97b92cc9451443ca102ee087b7b0a3b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:28.948312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-00a3af2e-3572-4083-b59b-913da5c1e93e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b97b92cc9451443ca102ee087b7b0a3b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:28.956469 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-00a3af2e-3572-4083-b59b-913da5c1e93e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b97b92cc9451443ca102ee087b7b0a3b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:28.957378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-00a3af2e-3572-4083-b59b-913da5c1e93e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b97b92cc9451443ca102ee087b7b0a3b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:28.963875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1278/2541] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:28 2022] DELETE /identity/v3/users/b97b92cc9451443ca102ee087b7b0a3b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:28.968594 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8364e6-6d24-472b-9344-11bb1a44e6ca None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:28.969501 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8364e6-6d24-472b-9344-11bb1a44e6ca None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:28.969785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8364e6-6d24-472b-9344-11bb1a44e6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:28.989136 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-8e8364e6-6d24-472b-9344-11bb1a44e6ca None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:28.990878 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8e8364e6-6d24-472b-9344-11bb1a44e6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:29.060091 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1264/2542] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:29.070201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3af2e-3572-4083-b59b-913da5c1e93e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:29.071842 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-00a3af2e-3572-4083-b59b-913da5c1e93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:29.129145 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3af2e-3572-4083-b59b-913da5c1e93e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:29.133629 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485f215-db8d-48da-a6a7-9c2a98591b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:29.134235 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485f215-db8d-48da-a6a7-9c2a98591b0d service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:29.134508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485f215-db8d-48da-a6a7-9c2a98591b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:29.134905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485f215-db8d-48da-a6a7-9c2a98591b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:29.145118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485f215-db8d-48da-a6a7-9c2a98591b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:29.146615 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485f215-db8d-48da-a6a7-9c2a98591b0d service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:29.156714 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1279/2543] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:29.552790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8364e6-6d24-472b-9344-11bb1a44e6ca None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:29.554715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8e8364e6-6d24-472b-9344-11bb1a44e6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:29.567730 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8364e6-6d24-472b-9344-11bb1a44e6ca None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:29.576343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcf615-90aa-4c6a-9258-32cd41571c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:29.577306 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcf615-90aa-4c6a-9258-32cd41571c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:29.577957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcf615-90aa-4c6a-9258-32cd41571c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:29.578578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcf615-90aa-4c6a-9258-32cd41571c98 admin admin] PATH_INFO: `/v3/projects/cecbca8dd4944b93836e910fa7f822ac` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:29.581194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcf615-90aa-4c6a-9258-32cd41571c98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cecbca8dd4944b93836e910fa7f822ac)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:29.584241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcf615-90aa-4c6a-9258-32cd41571c98 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:29.648055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6cfcf615-90aa-4c6a-9258-32cd41571c98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cecbca8dd4944b93836e910fa7f822ac', 'request_id': 'req-6cfcf615-90aa-4c6a-9258-32cd41571c98'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:29.654279 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6cfcf615-90aa-4c6a-9258-32cd41571c98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cecbca8dd4944b93836e910fa7f822ac', 'request_id': 'req-6cfcf615-90aa-4c6a-9258-32cd41571c98'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:29.661505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6cfcf615-90aa-4c6a-9258-32cd41571c98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cecbca8dd4944b93836e910fa7f822ac', 'request_id': 'req-6cfcf615-90aa-4c6a-9258-32cd41571c98'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:29.674363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6cfcf615-90aa-4c6a-9258-32cd41571c98 admin admin] The token cache is being invalidate because project cecbca8dd4944b93836e910fa7f822ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:29.674664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6cfcf615-90aa-4c6a-9258-32cd41571c98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:29.675437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-6cfcf615-90aa-4c6a-9258-32cd41571c98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:29.676982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1265/2544] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:29 2022] DELETE /identity/v3/projects/cecbca8dd4944b93836e910fa7f822ac => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:29.682445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9efe9-3c81-4431-890a-1c409ab16c4b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:29.682737 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9efe9-3c81-4431-890a-1c409ab16c4b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:29.683016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9efe9-3c81-4431-890a-1c409ab16c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:29.707314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7cc9efe9-3c81-4431-890a-1c409ab16c4b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:29.708739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7cc9efe9-3c81-4431-890a-1c409ab16c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:29.772749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1280/2545] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:29.781038 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32805353-e26b-4af8-8577-250cb82a307d None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:29.781336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32805353-e26b-4af8-8577-250cb82a307d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:29.781647 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32805353-e26b-4af8-8577-250cb82a307d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:29.801284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-32805353-e26b-4af8-8577-250cb82a307d None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:29.803027 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-32805353-e26b-4af8-8577-250cb82a307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:29.811261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1266/2546] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:29.815765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9efe9-3c81-4431-890a-1c409ab16c4b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:29.826436 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9efe9-3c81-4431-890a-1c409ab16c4b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:29.831021 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284086fe-2793-45ca-843c-a8e397772e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:29.831713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284086fe-2793-45ca-843c-a8e397772e5d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:29.831989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284086fe-2793-45ca-843c-a8e397772e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:29.832265 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284086fe-2793-45ca-843c-a8e397772e5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:29.832907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284086fe-2793-45ca-843c-a8e397772e5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:34:29.833213 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284086fe-2793-45ca-843c-a8e397772e5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:29.834826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284086fe-2793-45ca-843c-a8e397772e5d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:29.848240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1281/2547] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:34:29.853014 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32805353-e26b-4af8-8577-250cb82a307d None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:29.859154 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:29.863199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32805353-e26b-4af8-8577-250cb82a307d None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:29.870101 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1418d4-971e-4709-9c9b-81d3b7639e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:29.870876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1418d4-971e-4709-9c9b-81d3b7639e83 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:29.871331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1418d4-971e-4709-9c9b-81d3b7639e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:29.871813 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1418d4-971e-4709-9c9b-81d3b7639e83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:29.872570 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1418d4-971e-4709-9c9b-81d3b7639e83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:29.874554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1418d4-971e-4709-9c9b-81d3b7639e83 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:29.903483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1267/2548] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:29 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:29.908224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284086fe-2793-45ca-843c-a8e397772e5d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:29.934778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284086fe-2793-45ca-843c-a8e397772e5d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:29.970264 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:29.996018 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1eb96b-6b3f-4337-8abf-6875d4f6e6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:29.996627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1eb96b-6b3f-4337-8abf-6875d4f6e6ef admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:29.996905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1eb96b-6b3f-4337-8abf-6875d4f6e6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:29.997170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1eb96b-6b3f-4337-8abf-6875d4f6e6ef admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:29.997809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1eb96b-6b3f-4337-8abf-6875d4f6e6ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:29.999750 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1eb96b-6b3f-4337-8abf-6875d4f6e6ef admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:30.027298 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1282/2549] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:29 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:30.028643 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1418d4-971e-4709-9c9b-81d3b7639e83 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:30.041183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1418d4-971e-4709-9c9b-81d3b7639e83 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:30.045842 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bbe55-5171-4451-b33c-dc0dd33ceab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:30.046440 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bbe55-5171-4451-b33c-dc0dd33ceab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.046715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bbe55-5171-4451-b33c-dc0dd33ceab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.046986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bbe55-5171-4451-b33c-dc0dd33ceab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.047814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bbe55-5171-4451-b33c-dc0dd33ceab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:30.049663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bbe55-5171-4451-b33c-dc0dd33ceab6 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:30.060729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1268/2550] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:34:30.064610 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1eb96b-6b3f-4337-8abf-6875d4f6e6ef admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:30.075135 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1eb96b-6b3f-4337-8abf-6875d4f6e6ef admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:30.084273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1c53d-b185-4102-aeb3-49764bcbbd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:30.084918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1c53d-b185-4102-aeb3-49764bcbbd34 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.085203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1c53d-b185-4102-aeb3-49764bcbbd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.085482 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1c53d-b185-4102-aeb3-49764bcbbd34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.086137 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1c53d-b185-4102-aeb3-49764bcbbd34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:30.087791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1c53d-b185-4102-aeb3-49764bcbbd34 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:30.099023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1283/2551] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:30.103561 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bbe55-5171-4451-b33c-dc0dd33ceab6 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:30.113604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bbe55-5171-4451-b33c-dc0dd33ceab6 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:30.119529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c2785-b5ff-44ff-8b37-d9094be20ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:30.120449 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c2785-b5ff-44ff-8b37-d9094be20ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.120768 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c2785-b5ff-44ff-8b37-d9094be20ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.121138 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c2785-b5ff-44ff-8b37-d9094be20ccb admin admin] PATH_INFO: `/v3/projects/959990a759224e47b146386f4f939a79/users/2edeae76227b4d72acff2f4434e2cd8b/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.137876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c2785-b5ff-44ff-8b37-d9094be20ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=959990a759224e47b146386f4f939a79, user_id=2edeae76227b4d72acff2f4434e2cd8b, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:30.141394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c2785-b5ff-44ff-8b37-d9094be20ccb admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:30.147481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1c53d-b185-4102-aeb3-49764bcbbd34 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:30.150453 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-14b1c53d-b185-4102-aeb3-49764bcbbd34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:30.163153 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1269/2552] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:30 2022] PUT /identity/v3/projects/959990a759224e47b146386f4f939a79/users/2edeae76227b4d72acff2f4434e2cd8b/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:30.169925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1c53d-b185-4102-aeb3-49764bcbbd34 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:30.174695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124d53d-ab2a-4d15-ad37-d3ecbc2d9dac None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.174981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124d53d-ab2a-4d15-ad37-d3ecbc2d9dac None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.175251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124d53d-ab2a-4d15-ad37-d3ecbc2d9dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.203407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-6124d53d-ab2a-4d15-ad37-d3ecbc2d9dac None None] MFA Rules not processed for user `2edeae76227b4d72acff2f4434e2cd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:30.204640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6124d53d-ab2a-4d15-ad37-d3ecbc2d9dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:30.228294 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b82b6-b234-404b-9447-ae80ff44b540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:30.228946 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b82b6-b234-404b-9447-ae80ff44b540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.229228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b82b6-b234-404b-9447-ae80ff44b540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.229504 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b82b6-b234-404b-9447-ae80ff44b540 admin admin] PATH_INFO: `/v3/users/715d0dc80f204a0fb1f8db5da40003ca` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.230612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b82b6-b234-404b-9447-ae80ff44b540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=715d0dc80f204a0fb1f8db5da40003ca)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:30.232559 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b82b6-b234-404b-9447-ae80ff44b540 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:30.269567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1270/2553] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:30.274126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124d53d-ab2a-4d15-ad37-d3ecbc2d9dac None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:30.282136 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e1b82b6-b234-404b-9447-ae80ff44b540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '715d0dc80f204a0fb1f8db5da40003ca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.285743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124d53d-ab2a-4d15-ad37-d3ecbc2d9dac None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:30.288960 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e1b82b6-b234-404b-9447-ae80ff44b540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '715d0dc80f204a0fb1f8db5da40003ca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.290778 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b011978-f42e-4b51-b878-e036c0c1888c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:30.291459 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b011978-f42e-4b51-b878-e036c0c1888c admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.291768 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b011978-f42e-4b51-b878-e036c0c1888c admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.292044 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b011978-f42e-4b51-b878-e036c0c1888c admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.292664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b011978-f42e-4b51-b878-e036c0c1888c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:30.294560 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b011978-f42e-4b51-b878-e036c0c1888c admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:30.301485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e1b82b6-b234-404b-9447-ae80ff44b540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '715d0dc80f204a0fb1f8db5da40003ca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.309917 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e1b82b6-b234-404b-9447-ae80ff44b540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '715d0dc80f204a0fb1f8db5da40003ca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.310688 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e1b82b6-b234-404b-9447-ae80ff44b540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '715d0dc80f204a0fb1f8db5da40003ca'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.317623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1284/2554] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:30 2022] DELETE /identity/v3/users/715d0dc80f204a0fb1f8db5da40003ca => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:34:30.326988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1271/2555] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:30 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:34:30.328180 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b82b6-b234-404b-9447-ae80ff44b540 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:30.329562 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2e1b82b6-b234-404b-9447-ae80ff44b540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:30.374154 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:30.398277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b82b6-b234-404b-9447-ae80ff44b540 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:30.454864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c0069-6246-41a9-b449-078528a8ee42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:30.455473 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c0069-6246-41a9-b449-078528a8ee42 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.455779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c0069-6246-41a9-b449-078528a8ee42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.456051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c0069-6246-41a9-b449-078528a8ee42 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.456672 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c0069-6246-41a9-b449-078528a8ee42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:30.459148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c0069-6246-41a9-b449-078528a8ee42 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:30.483712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1285/2556] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:30 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:30.487591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b011978-f42e-4b51-b878-e036c0c1888c admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:30.498971 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b011978-f42e-4b51-b878-e036c0c1888c admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:30.504324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244be5f-504b-48fa-a4e9-f04c07b002f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:30.504589 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c0069-6246-41a9-b449-078528a8ee42 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:30.504764 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244be5f-504b-48fa-a4e9-f04c07b002f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.505008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244be5f-504b-48fa-a4e9-f04c07b002f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.505421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244be5f-504b-48fa-a4e9-f04c07b002f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.506073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244be5f-504b-48fa-a4e9-f04c07b002f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:30.506317 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2e9c0069-6246-41a9-b449-078528a8ee42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:30.507835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244be5f-504b-48fa-a4e9-f04c07b002f6 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:30.519617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1272/2557] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:30.520312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c0069-6246-41a9-b449-078528a8ee42 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:30.526240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:30.526893 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.527169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.527442 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] PATH_INFO: `/v3/projects/8655dc481389478a9e3a78929dbf11aa` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.528305 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244be5f-504b-48fa-a4e9-f04c07b002f6 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:30.528589 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] RBAC: Authorizing `identity:delete_project(project_id=8655dc481389478a9e3a78929dbf11aa)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:30.530731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:30.539893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244be5f-504b-48fa-a4e9-f04c07b002f6 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:30.544613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a2c42-b72d-4bca-9b85-7f8fae9ffd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:30.545257 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2c42-b72d-4bca-9b85-7f8fae9ffd9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.545539 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2c42-b72d-4bca-9b85-7f8fae9ffd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.545815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2c42-b72d-4bca-9b85-7f8fae9ffd9f admin admin] PATH_INFO: `/v3/projects/d0ce4d30056f48d1bccc59d8f894248b/users/437dc85e8ed6472fa4137780b43b70b6/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.561671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a2c42-b72d-4bca-9b85-7f8fae9ffd9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ce4d30056f48d1bccc59d8f894248b, user_id=437dc85e8ed6472fa4137780b43b70b6, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:30.564415 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a2c42-b72d-4bca-9b85-7f8fae9ffd9f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:30.576965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1273/2558] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:30 2022] PUT /identity/v3/projects/d0ce4d30056f48d1bccc59d8f894248b/users/437dc85e8ed6472fa4137780b43b70b6/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:30.578905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8655dc481389478a9e3a78929dbf11aa', 'request_id': 'req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.581133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a2c42-b72d-4bca-9b85-7f8fae9ffd9f admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:30.584961 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8655dc481389478a9e3a78929dbf11aa', 'request_id': 'req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.593070 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8655dc481389478a9e3a78929dbf11aa', 'request_id': 'req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.605922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] The token cache is being invalidate because project 8655dc481389478a9e3a78929dbf11aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:30.606211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.606872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.608595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1286/2559] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:30 2022] DELETE /identity/v3/projects/8655dc481389478a9e3a78929dbf11aa => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:34:30.612209 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:30.613511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:30.628214 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:30.655609 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a2c42-b72d-4bca-9b85-7f8fae9ffd9f admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:30.660086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664d9e2-c8ca-4eba-9584-bc85a5a8a2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:30.660706 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664d9e2-c8ca-4eba-9584-bc85a5a8a2fc admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.660986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664d9e2-c8ca-4eba-9584-bc85a5a8a2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.661250 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664d9e2-c8ca-4eba-9584-bc85a5a8a2fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.662552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664d9e2-c8ca-4eba-9584-bc85a5a8a2fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:30.664435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664d9e2-c8ca-4eba-9584-bc85a5a8a2fc admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:30.675655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1274/2560] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:34:30.679692 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664d9e2-c8ca-4eba-9584-bc85a5a8a2fc admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:30.681360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6664d9e2-c8ca-4eba-9584-bc85a5a8a2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:30.683093 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:30.694019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e315d-91ce-40b3-b705-1d0e7ce1d4be admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:30.699716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664d9e2-c8ca-4eba-9584-bc85a5a8a2fc admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:30.700671 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22050d99-526e-457f-a7ef-7999ca8107c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:30.701475 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22050d99-526e-457f-a7ef-7999ca8107c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.701795 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22050d99-526e-457f-a7ef-7999ca8107c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.702069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22050d99-526e-457f-a7ef-7999ca8107c5 admin admin] PATH_INFO: `/v3/users/8de67995f78840eb94afe9fe4d86e3f8` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.703186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22050d99-526e-457f-a7ef-7999ca8107c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8de67995f78840eb94afe9fe4d86e3f8)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:30.705018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d300d01-7c9d-46bd-8720-750dbd5353de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:30.705161 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22050d99-526e-457f-a7ef-7999ca8107c5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:30.705667 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d300d01-7c9d-46bd-8720-750dbd5353de admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.705947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d300d01-7c9d-46bd-8720-750dbd5353de admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.706220 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d300d01-7c9d-46bd-8720-750dbd5353de admin admin] PATH_INFO: `/v3/projects/d0ce4d30056f48d1bccc59d8f894248b/users/437dc85e8ed6472fa4137780b43b70b6/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.708157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d300d01-7c9d-46bd-8720-750dbd5353de admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ce4d30056f48d1bccc59d8f894248b, user_id=437dc85e8ed6472fa4137780b43b70b6, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:30.712256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d300d01-7c9d-46bd-8720-750dbd5353de admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:30.724288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1275/2561] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:30 2022] PUT /identity/v3/projects/d0ce4d30056f48d1bccc59d8f894248b/users/437dc85e8ed6472fa4137780b43b70b6/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:34:30.734351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d0952-52b1-4598-847f-b7abb489c4ed None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.734740 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d0952-52b1-4598-847f-b7abb489c4ed None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.735037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d0952-52b1-4598-847f-b7abb489c4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.760968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-22050d99-526e-457f-a7ef-7999ca8107c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8de67995f78840eb94afe9fe4d86e3f8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.768234 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-9c4d0952-52b1-4598-847f-b7abb489c4ed None None] MFA Rules not processed for user `437dc85e8ed6472fa4137780b43b70b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:30.768433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-22050d99-526e-457f-a7ef-7999ca8107c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8de67995f78840eb94afe9fe4d86e3f8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.769478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9c4d0952-52b1-4598-847f-b7abb489c4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:30.782791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-22050d99-526e-457f-a7ef-7999ca8107c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8de67995f78840eb94afe9fe4d86e3f8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.792146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-22050d99-526e-457f-a7ef-7999ca8107c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8de67995f78840eb94afe9fe4d86e3f8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.792907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-22050d99-526e-457f-a7ef-7999ca8107c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8de67995f78840eb94afe9fe4d86e3f8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:30.816133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1287/2562] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:30 2022] DELETE /identity/v3/users/8de67995f78840eb94afe9fe4d86e3f8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:34:30.865199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1276/2563] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:34:30.876566 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22050d99-526e-457f-a7ef-7999ca8107c5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:30.878026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-22050d99-526e-457f-a7ef-7999ca8107c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:30.952816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22050d99-526e-457f-a7ef-7999ca8107c5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:30.957928 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979135e-8a03-4f47-94ca-2e3fcbf29a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:30.959494 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979135e-8a03-4f47-94ca-2e3fcbf29a7b service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:30.959954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979135e-8a03-4f47-94ca-2e3fcbf29a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:30.960376 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979135e-8a03-4f47-94ca-2e3fcbf29a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:30.992987 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d0952-52b1-4598-847f-b7abb489c4ed None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:30.994999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9c4d0952-52b1-4598-847f-b7abb489c4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:31.039959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979135e-8a03-4f47-94ca-2e3fcbf29a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:31.041645 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979135e-8a03-4f47-94ca-2e3fcbf29a7b service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:31.058500 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1288/2564] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:31.074828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d0952-52b1-4598-847f-b7abb489c4ed None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:31.142153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979135e-8a03-4f47-94ca-2e3fcbf29a7b service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:31.142891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:31.151835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979135e-8a03-4f47-94ca-2e3fcbf29a7b service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:31.152334 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ec153-8656-49c3-ac94-b3e42674cc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:31.153126 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ec153-8656-49c3-ac94-b3e42674cc64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:31.153513 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ec153-8656-49c3-ac94-b3e42674cc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:31.153883 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ec153-8656-49c3-ac94-b3e42674cc64 admin admin] PATH_INFO: `/v3/projects/2e5c38695132459eb597cef9cbc5c8b6` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:31.155064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ec153-8656-49c3-ac94-b3e42674cc64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e5c38695132459eb597cef9cbc5c8b6)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:31.155705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0653d-d17b-4a90-9e2d-b804a4d4cfdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:31.156324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0653d-d17b-4a90-9e2d-b804a4d4cfdd service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:31.156603 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0653d-d17b-4a90-9e2d-b804a4d4cfdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:31.156876 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0653d-d17b-4a90-9e2d-b804a4d4cfdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:31.157151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ec153-8656-49c3-ac94-b3e42674cc64 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:31.158639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8ec0653d-d17b-4a90-9e2d-b804a4d4cfdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:31.182902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:31.207931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5a4ec153-8656-49c3-ac94-b3e42674cc64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e5c38695132459eb597cef9cbc5c8b6', 'request_id': 'req-5a4ec153-8656-49c3-ac94-b3e42674cc64'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:31.213697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5a4ec153-8656-49c3-ac94-b3e42674cc64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e5c38695132459eb597cef9cbc5c8b6', 'request_id': 'req-5a4ec153-8656-49c3-ac94-b3e42674cc64'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:31.220411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5a4ec153-8656-49c3-ac94-b3e42674cc64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e5c38695132459eb597cef9cbc5c8b6', 'request_id': 'req-5a4ec153-8656-49c3-ac94-b3e42674cc64'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:31.225753 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0653d-d17b-4a90-9e2d-b804a4d4cfdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:31.227176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0653d-d17b-4a90-9e2d-b804a4d4cfdd service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:31.231538 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5a4ec153-8656-49c3-ac94-b3e42674cc64 admin admin] The token cache is being invalidate because project 2e5c38695132459eb597cef9cbc5c8b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:31.231923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5a4ec153-8656-49c3-ac94-b3e42674cc64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:31.232552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-5a4ec153-8656-49c3-ac94-b3e42674cc64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:31.234034 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1277/2565] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:30 2022] DELETE /identity/v3/projects/2e5c38695132459eb597cef9cbc5c8b6 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:34:31.237701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1289/2566] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:34:31.238674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce94f0f-d3e0-463a-a216-9a88818d94de None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:31.238976 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce94f0f-d3e0-463a-a216-9a88818d94de None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:31.239238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce94f0f-d3e0-463a-a216-9a88818d94de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:31.260776 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-2ce94f0f-d3e0-463a-a216-9a88818d94de None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:31.261989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2ce94f0f-d3e0-463a-a216-9a88818d94de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:31.328588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1278/2567] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:34:31.336908 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb409320-de4b-4fc5-a280-b3ebae543646 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:31.336908 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb409320-de4b-4fc5-a280-b3ebae543646 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:31.337219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb409320-de4b-4fc5-a280-b3ebae543646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:31.372056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-eb409320-de4b-4fc5-a280-b3ebae543646 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:31.373491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eb409320-de4b-4fc5-a280-b3ebae543646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:31.384718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1290/2568] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:31.391819 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce94f0f-d3e0-463a-a216-9a88818d94de None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:31.412335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce94f0f-d3e0-463a-a216-9a88818d94de None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:31.418142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62b54e-389b-46b0-a6fb-8c9c5169c32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:31.418873 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62b54e-389b-46b0-a6fb-8c9c5169c32a admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:31.419216 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62b54e-389b-46b0-a6fb-8c9c5169c32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:31.419552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62b54e-389b-46b0-a6fb-8c9c5169c32a admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:31.420326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62b54e-389b-46b0-a6fb-8c9c5169c32a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:34:31.420701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62b54e-389b-46b0-a6fb-8c9c5169c32a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:31.422460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62b54e-389b-46b0-a6fb-8c9c5169c32a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:31.439524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1279/2569] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:34:31.440824 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb409320-de4b-4fc5-a280-b3ebae543646 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:31.451250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb409320-de4b-4fc5-a280-b3ebae543646 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:31.455965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1ba33-82e4-484c-8e43-1a52771b0902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:31.456571 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1ba33-82e4-484c-8e43-1a52771b0902 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:31.456845 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1ba33-82e4-484c-8e43-1a52771b0902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:31.457116 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1ba33-82e4-484c-8e43-1a52771b0902 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:31.457746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1ba33-82e4-484c-8e43-1a52771b0902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:31.459715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1ba33-82e4-484c-8e43-1a52771b0902 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:31.489012 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1291/2570] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:31 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:31.493853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62b54e-389b-46b0-a6fb-8c9c5169c32a admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:31.504062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62b54e-389b-46b0-a6fb-8c9c5169c32a admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:31.562730 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee3919-c88f-4dd2-8005-0ad66a948a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:31.563340 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee3919-c88f-4dd2-8005-0ad66a948a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:31.563651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee3919-c88f-4dd2-8005-0ad66a948a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:31.563928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee3919-c88f-4dd2-8005-0ad66a948a45 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:31.564567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee3919-c88f-4dd2-8005-0ad66a948a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:31.566525 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee3919-c88f-4dd2-8005-0ad66a948a45 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:31.587420 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1280/2571] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:31 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:34:31.591563 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1ba33-82e4-484c-8e43-1a52771b0902 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:31.602100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1ba33-82e4-484c-8e43-1a52771b0902 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:31.606634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b462af-5051-4548-a458-d0d589b92d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:31.607218 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b462af-5051-4548-a458-d0d589b92d23 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:31.607498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b462af-5051-4548-a458-d0d589b92d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:31.607799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b462af-5051-4548-a458-d0d589b92d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:31.608434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b462af-5051-4548-a458-d0d589b92d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:31.610009 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b462af-5051-4548-a458-d0d589b92d23 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:31.620655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1292/2572] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:31.624486 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee3919-c88f-4dd2-8005-0ad66a948a45 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:31.629728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:31.636116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee3919-c88f-4dd2-8005-0ad66a948a45 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:31.642118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a799d8-f690-4c6f-bab4-75fb804897dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:31.642695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a799d8-f690-4c6f-bab4-75fb804897dd admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:31.642981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a799d8-f690-4c6f-bab4-75fb804897dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:31.643317 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a799d8-f690-4c6f-bab4-75fb804897dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:31.644371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a799d8-f690-4c6f-bab4-75fb804897dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:31.645587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a799d8-f690-4c6f-bab4-75fb804897dd admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:31.658233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1281/2573] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:31.661257 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b462af-5051-4548-a458-d0d589b92d23 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:31.671733 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b462af-5051-4548-a458-d0d589b92d23 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:31.676349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a9f2c-7992-4900-bf1d-25fce0bd8ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:31.676981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a9f2c-7992-4900-bf1d-25fce0bd8ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:31.677407 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a9f2c-7992-4900-bf1d-25fce0bd8ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:31.678211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a9f2c-7992-4900-bf1d-25fce0bd8ad4 admin admin] PATH_INFO: `/v3/projects/8b13e67dbbf147f4a6de198dba038d72/users/37cd6d62877f4b5181759c709ee68c0b/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:31.692593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a9f2c-7992-4900-bf1d-25fce0bd8ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b13e67dbbf147f4a6de198dba038d72, user_id=37cd6d62877f4b5181759c709ee68c0b, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:31.696196 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a9f2c-7992-4900-bf1d-25fce0bd8ad4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:31.708632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1293/2574] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:31 2022] PUT /identity/v3/projects/8b13e67dbbf147f4a6de198dba038d72/users/37cd6d62877f4b5181759c709ee68c0b/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:31.719849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fec60-1b4e-4b8f-b585-e024b50372a1 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:31.720144 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fec60-1b4e-4b8f-b585-e024b50372a1 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:31.720422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fec60-1b4e-4b8f-b585-e024b50372a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:31.756399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-8b9fec60-1b4e-4b8f-b585-e024b50372a1 None None] MFA Rules not processed for user `37cd6d62877f4b5181759c709ee68c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:31.757637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8b9fec60-1b4e-4b8f-b585-e024b50372a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:31.829186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1282/2575] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:31.837082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a9f2c-7992-4900-bf1d-25fce0bd8ad4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:31.838752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-367a9f2c-7992-4900-bf1d-25fce0bd8ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:31.898412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a9f2c-7992-4900-bf1d-25fce0bd8ad4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:31.902305 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24b353-fe6b-4adf-a12a-1369a71eb929 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:31.902910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24b353-fe6b-4adf-a12a-1369a71eb929 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:31.903201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24b353-fe6b-4adf-a12a-1369a71eb929 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:31.903459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24b353-fe6b-4adf-a12a-1369a71eb929 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:31.912918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24b353-fe6b-4adf-a12a-1369a71eb929 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:31.914499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24b353-fe6b-4adf-a12a-1369a71eb929 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:31.924483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1294/2576] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:31.946755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fec60-1b4e-4b8f-b585-e024b50372a1 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:32.012496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fec60-1b4e-4b8f-b585-e024b50372a1 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:32.017026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b108ff0-0aa3-4b69-9b52-d7caf3e05b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:32.017646 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b108ff0-0aa3-4b69-9b52-d7caf3e05b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:32.017923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b108ff0-0aa3-4b69-9b52-d7caf3e05b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:32.018193 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b108ff0-0aa3-4b69-9b52-d7caf3e05b0a admin admin] PATH_INFO: `/v3/users/37cd6d62877f4b5181759c709ee68c0b` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:32.019310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b108ff0-0aa3-4b69-9b52-d7caf3e05b0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=37cd6d62877f4b5181759c709ee68c0b)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:32.021770 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b108ff0-0aa3-4b69-9b52-d7caf3e05b0a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:32.065572 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4b108ff0-0aa3-4b69-9b52-d7caf3e05b0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37cd6d62877f4b5181759c709ee68c0b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:32.077271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4b108ff0-0aa3-4b69-9b52-d7caf3e05b0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37cd6d62877f4b5181759c709ee68c0b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:32.084932 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4b108ff0-0aa3-4b69-9b52-d7caf3e05b0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37cd6d62877f4b5181759c709ee68c0b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:32.085556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4b108ff0-0aa3-4b69-9b52-d7caf3e05b0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37cd6d62877f4b5181759c709ee68c0b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:32.092283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-4b108ff0-0aa3-4b69-9b52-d7caf3e05b0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37cd6d62877f4b5181759c709ee68c0b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:32.094339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:32.099728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1283/2577] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:31 2022] DELETE /identity/v3/users/37cd6d62877f4b5181759c709ee68c0b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:32.104495 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793be1f-32ff-4108-946e-07275c52b056 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:32.104782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793be1f-32ff-4108-946e-07275c52b056 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:32.105060 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793be1f-32ff-4108-946e-07275c52b056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:32.123547 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5793be1f-32ff-4108-946e-07275c52b056 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:32.124781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5793be1f-32ff-4108-946e-07275c52b056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:32.183157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1295/2578] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:32.193307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b108ff0-0aa3-4b69-9b52-d7caf3e05b0a admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:32.194792 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4b108ff0-0aa3-4b69-9b52-d7caf3e05b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:32.249738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b108ff0-0aa3-4b69-9b52-d7caf3e05b0a admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:32.253393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1170017-4696-4933-b5cd-49e4de0bbe64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:32.253988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1170017-4696-4933-b5cd-49e4de0bbe64 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:32.254271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1170017-4696-4933-b5cd-49e4de0bbe64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:32.254546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1170017-4696-4933-b5cd-49e4de0bbe64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:32.264172 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1170017-4696-4933-b5cd-49e4de0bbe64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:32.265605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1170017-4696-4933-b5cd-49e4de0bbe64 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:32.275989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1284/2579] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:32.550583 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793be1f-32ff-4108-946e-07275c52b056 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:32.552030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5793be1f-32ff-4108-946e-07275c52b056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:32.564731 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:32.580740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793be1f-32ff-4108-946e-07275c52b056 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:32.586607 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eeefe-c07c-421c-a172-eb5c975042a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:32.587240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eeefe-c07c-421c-a172-eb5c975042a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:32.588169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eeefe-c07c-421c-a172-eb5c975042a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:32.588169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eeefe-c07c-421c-a172-eb5c975042a8 admin admin] PATH_INFO: `/v3/projects/8b13e67dbbf147f4a6de198dba038d72` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:32.589073 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6eeefe-c07c-421c-a172-eb5c975042a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b13e67dbbf147f4a6de198dba038d72)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:32.590993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6eeefe-c07c-421c-a172-eb5c975042a8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:32.640293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-aa6eeefe-c07c-421c-a172-eb5c975042a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b13e67dbbf147f4a6de198dba038d72', 'request_id': 'req-aa6eeefe-c07c-421c-a172-eb5c975042a8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:32.646582 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-aa6eeefe-c07c-421c-a172-eb5c975042a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b13e67dbbf147f4a6de198dba038d72', 'request_id': 'req-aa6eeefe-c07c-421c-a172-eb5c975042a8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:32.654445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-aa6eeefe-c07c-421c-a172-eb5c975042a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b13e67dbbf147f4a6de198dba038d72', 'request_id': 'req-aa6eeefe-c07c-421c-a172-eb5c975042a8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:32.668213 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-aa6eeefe-c07c-421c-a172-eb5c975042a8 admin admin] The token cache is being invalidate because project 8b13e67dbbf147f4a6de198dba038d72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:32.668514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-aa6eeefe-c07c-421c-a172-eb5c975042a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:32.669288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-aa6eeefe-c07c-421c-a172-eb5c975042a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:32.670744 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1296/2580] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:32 2022] DELETE /identity/v3/projects/8b13e67dbbf147f4a6de198dba038d72 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:32.675799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c667a-c9c1-49d4-9ac5-fce6a766e795 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:32.676191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c667a-c9c1-49d4-9ac5-fce6a766e795 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:32.676533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c667a-c9c1-49d4-9ac5-fce6a766e795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:32.702436 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c37c667a-c9c1-49d4-9ac5-fce6a766e795 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:32.703655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c37c667a-c9c1-49d4-9ac5-fce6a766e795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:32.785052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1285/2581] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:34:32.786755 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90350e58-758c-4d14-8d39-28f95fd00a35 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:32.787670 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90350e58-758c-4d14-8d39-28f95fd00a35 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:32.788200 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90350e58-758c-4d14-8d39-28f95fd00a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:32.809771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-90350e58-758c-4d14-8d39-28f95fd00a35 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:32.810979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-90350e58-758c-4d14-8d39-28f95fd00a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:32.824397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c667a-c9c1-49d4-9ac5-fce6a766e795 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:32.825298 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1297/2582] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:32.834411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c667a-c9c1-49d4-9ac5-fce6a766e795 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:32.839407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c74dea-1387-4354-919d-4fff96fb3e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:32.840082 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c74dea-1387-4354-919d-4fff96fb3e9d admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:32.840403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c74dea-1387-4354-919d-4fff96fb3e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:32.840768 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c74dea-1387-4354-919d-4fff96fb3e9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:32.841504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c74dea-1387-4354-919d-4fff96fb3e9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:34:32.841880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c74dea-1387-4354-919d-4fff96fb3e9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:32.843624 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c74dea-1387-4354-919d-4fff96fb3e9d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:32.859374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1286/2583] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:34:32.860505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90350e58-758c-4d14-8d39-28f95fd00a35 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:32.870725 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90350e58-758c-4d14-8d39-28f95fd00a35 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:32.875415 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e697b-c71f-40e7-86a9-afd881e490b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:32.876065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e697b-c71f-40e7-86a9-afd881e490b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:32.876340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e697b-c71f-40e7-86a9-afd881e490b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:32.876610 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e697b-c71f-40e7-86a9-afd881e490b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:32.877247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e697b-c71f-40e7-86a9-afd881e490b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:32.879157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e697b-c71f-40e7-86a9-afd881e490b8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:32.907172 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1298/2584] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:32 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:34:32.917564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c74dea-1387-4354-919d-4fff96fb3e9d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:32.928823 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c74dea-1387-4354-919d-4fff96fb3e9d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:32.988061 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce95e78-2981-4f49-b6d3-1c1df7f6ea47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:32.988718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce95e78-2981-4f49-b6d3-1c1df7f6ea47 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:32.989023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce95e78-2981-4f49-b6d3-1c1df7f6ea47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:32.989331 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce95e78-2981-4f49-b6d3-1c1df7f6ea47 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:32.990552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce95e78-2981-4f49-b6d3-1c1df7f6ea47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:32.992778 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce95e78-2981-4f49-b6d3-1c1df7f6ea47 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:33.020721 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1287/2585] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:32 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:33.021946 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e697b-c71f-40e7-86a9-afd881e490b8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:33.036107 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e697b-c71f-40e7-86a9-afd881e490b8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:33.042909 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9ad17-217f-4f31-8545-9cca5748725e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:33.043740 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9ad17-217f-4f31-8545-9cca5748725e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:33.054531 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9ad17-217f-4f31-8545-9cca5748725e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:33.054824 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9ad17-217f-4f31-8545-9cca5748725e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:33.055545 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9ad17-217f-4f31-8545-9cca5748725e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:33.057618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9ad17-217f-4f31-8545-9cca5748725e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:33.069322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1299/2586] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:33.074119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce95e78-2981-4f49-b6d3-1c1df7f6ea47 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:33.086998 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce95e78-2981-4f49-b6d3-1c1df7f6ea47 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:33.092925 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b19ce-aea2-4b5d-931e-ceaa754b443a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:33.093564 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b19ce-aea2-4b5d-931e-ceaa754b443a admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:33.093843 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b19ce-aea2-4b5d-931e-ceaa754b443a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:33.094115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b19ce-aea2-4b5d-931e-ceaa754b443a admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:33.094881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b19ce-aea2-4b5d-931e-ceaa754b443a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:33.096594 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b19ce-aea2-4b5d-931e-ceaa754b443a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:33.109425 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1288/2587] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:34:33.113796 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9ad17-217f-4f31-8545-9cca5748725e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:33.125135 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9ad17-217f-4f31-8545-9cca5748725e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:33.129832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad116ad-f058-4564-a5e7-cbb006297b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:33.130551 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad116ad-f058-4564-a5e7-cbb006297b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:33.130828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad116ad-f058-4564-a5e7-cbb006297b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:33.131096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad116ad-f058-4564-a5e7-cbb006297b96 admin admin] PATH_INFO: `/v3/projects/00950b4875f54197bf44bd1ffa353937/users/dd6241594ed040ce85c4b017cc361894/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:33.146806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad116ad-f058-4564-a5e7-cbb006297b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00950b4875f54197bf44bd1ffa353937, user_id=dd6241594ed040ce85c4b017cc361894, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:33.149797 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad116ad-f058-4564-a5e7-cbb006297b96 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:33.165279 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1300/2588] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:33 2022] PUT /identity/v3/projects/00950b4875f54197bf44bd1ffa353937/users/dd6241594ed040ce85c4b017cc361894/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:33.170631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79288d-4a54-4b27-92b0-6ebbfa7204e8 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:33.171013 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79288d-4a54-4b27-92b0-6ebbfa7204e8 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:33.171361 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79288d-4a54-4b27-92b0-6ebbfa7204e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:33.193766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-2a79288d-4a54-4b27-92b0-6ebbfa7204e8 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:33.195125 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2a79288d-4a54-4b27-92b0-6ebbfa7204e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:33.199410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:33.258472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1289/2589] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:33.267832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad116ad-f058-4564-a5e7-cbb006297b96 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:33.269611 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cad116ad-f058-4564-a5e7-cbb006297b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:33.327959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad116ad-f058-4564-a5e7-cbb006297b96 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:33.331739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07f2db-757c-453c-b9a2-374534b35842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:33.332356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07f2db-757c-453c-b9a2-374534b35842 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:33.332640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07f2db-757c-453c-b9a2-374534b35842 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:33.332923 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07f2db-757c-453c-b9a2-374534b35842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:33.342735 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07f2db-757c-453c-b9a2-374534b35842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:33.344210 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07f2db-757c-453c-b9a2-374534b35842 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:33.354385 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1301/2590] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:37.489550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1e5df-c90f-4ba0-b6d7-66ba9b9e5b24 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:37.490033 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1e5df-c90f-4ba0-b6d7-66ba9b9e5b24 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:37.490117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1e5df-c90f-4ba0-b6d7-66ba9b9e5b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:37.536060 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-3de1e5df-c90f-4ba0-b6d7-66ba9b9e5b24 None None] MFA Rules not processed for user `dd6241594ed040ce85c4b017cc361894`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:37.537701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3de1e5df-c90f-4ba0-b6d7-66ba9b9e5b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:37.619840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1290/2591] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:37.625135 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07f2db-757c-453c-b9a2-374534b35842 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:37.638913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07f2db-757c-453c-b9a2-374534b35842 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:37.644775 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b92210-cac7-4d75-ac0d-1546cc54433f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:37.645786 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b92210-cac7-4d75-ac0d-1546cc54433f admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:37.646478 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b92210-cac7-4d75-ac0d-1546cc54433f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:37.647089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b92210-cac7-4d75-ac0d-1546cc54433f admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:37.647985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b92210-cac7-4d75-ac0d-1546cc54433f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:37.650620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b92210-cac7-4d75-ac0d-1546cc54433f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:37.690450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1302/2592] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:37 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:37.691124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1e5df-c90f-4ba0-b6d7-66ba9b9e5b24 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:37.713334 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1e5df-c90f-4ba0-b6d7-66ba9b9e5b24 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:37.809379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffecca-48ee-4cd7-b2f5-c1abf56c566a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:37.812228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffecca-48ee-4cd7-b2f5-c1abf56c566a admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:37.812697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffecca-48ee-4cd7-b2f5-c1abf56c566a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:37.813170 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffecca-48ee-4cd7-b2f5-c1abf56c566a admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:37.813995 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffecca-48ee-4cd7-b2f5-c1abf56c566a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:37.816175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffecca-48ee-4cd7-b2f5-c1abf56c566a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:37.843787 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1291/2593] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:37 2022] POST /identity/v3/users => generated 299 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:34:37.846066 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b92210-cac7-4d75-ac0d-1546cc54433f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:37.852365 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:37.862079 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b92210-cac7-4d75-ac0d-1546cc54433f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:37.869971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98022906-dfc7-4d4f-9823-71b8d6367187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:37.871017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98022906-dfc7-4d4f-9823-71b8d6367187 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:37.871523 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98022906-dfc7-4d4f-9823-71b8d6367187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:37.871837 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98022906-dfc7-4d4f-9823-71b8d6367187 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:37.872792 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98022906-dfc7-4d4f-9823-71b8d6367187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:37.874453 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98022906-dfc7-4d4f-9823-71b8d6367187 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:37.890843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1303/2594] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:37.893520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffecca-48ee-4cd7-b2f5-c1abf56c566a admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:37.904606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffecca-48ee-4cd7-b2f5-c1abf56c566a admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:37.909572 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec2371-93e6-4254-925a-0a487c25dca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:37.910579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec2371-93e6-4254-925a-0a487c25dca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:37.910857 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec2371-93e6-4254-925a-0a487c25dca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:37.911128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec2371-93e6-4254-925a-0a487c25dca2 admin admin] PATH_INFO: `/v3/projects/87e1811735e0433e88857ab3b3726533/users/d587495071784b1da16a0d771b5bfc9e/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:37.931918 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec2371-93e6-4254-925a-0a487c25dca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e1811735e0433e88857ab3b3726533, user_id=d587495071784b1da16a0d771b5bfc9e, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:37.934825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec2371-93e6-4254-925a-0a487c25dca2 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:37.952423 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1292/2595] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:37 2022] PUT /identity/v3/projects/87e1811735e0433e88857ab3b3726533/users/d587495071784b1da16a0d771b5bfc9e/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:37.957600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98022906-dfc7-4d4f-9823-71b8d6367187 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:37.968491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98022906-dfc7-4d4f-9823-71b8d6367187 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:38.030806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d73d01-afe3-43fa-b782-75f4d79cf6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:38.031442 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d73d01-afe3-43fa-b782-75f4d79cf6ce admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:38.031849 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d73d01-afe3-43fa-b782-75f4d79cf6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:38.032131 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d73d01-afe3-43fa-b782-75f4d79cf6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:38.033047 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d73d01-afe3-43fa-b782-75f4d79cf6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:38.034684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d73d01-afe3-43fa-b782-75f4d79cf6ce admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:38.045764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1304/2596] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:38.050022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec2371-93e6-4254-925a-0a487c25dca2 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:38.057470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:38.061634 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec2371-93e6-4254-925a-0a487c25dca2 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:38.068344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73840efd-61d8-4740-84e6-cb97fbfe194c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:38.069005 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73840efd-61d8-4740-84e6-cb97fbfe194c admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:38.069283 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73840efd-61d8-4740-84e6-cb97fbfe194c admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:38.069556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73840efd-61d8-4740-84e6-cb97fbfe194c admin admin] PATH_INFO: `/v3/projects/87e1811735e0433e88857ab3b3726533/users/d587495071784b1da16a0d771b5bfc9e/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:38.071497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73840efd-61d8-4740-84e6-cb97fbfe194c admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e1811735e0433e88857ab3b3726533, user_id=d587495071784b1da16a0d771b5bfc9e, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:38.074208 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73840efd-61d8-4740-84e6-cb97fbfe194c admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:38.084755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1293/2597] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:38 2022] PUT /identity/v3/projects/87e1811735e0433e88857ab3b3726533/users/d587495071784b1da16a0d771b5bfc9e/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:40.467668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d73d01-afe3-43fa-b782-75f4d79cf6ce admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:40.470506 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-57d73d01-afe3-43fa-b782-75f4d79cf6ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:40.490916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d73d01-afe3-43fa-b782-75f4d79cf6ce admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:40.613094 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04dd28-270a-4fd5-9375-915b955f2301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:40.615674 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04dd28-270a-4fd5-9375-915b955f2301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:40.616310 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04dd28-270a-4fd5-9375-915b955f2301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:40.617168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04dd28-270a-4fd5-9375-915b955f2301 admin admin] PATH_INFO: `/v3/users/2edeae76227b4d72acff2f4434e2cd8b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:40.619754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04dd28-270a-4fd5-9375-915b955f2301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2edeae76227b4d72acff2f4434e2cd8b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:40.624232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04dd28-270a-4fd5-9375-915b955f2301 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:40.716743 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b04dd28-270a-4fd5-9375-915b955f2301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2edeae76227b4d72acff2f4434e2cd8b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:40.726275 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b04dd28-270a-4fd5-9375-915b955f2301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2edeae76227b4d72acff2f4434e2cd8b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:40.742326 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b04dd28-270a-4fd5-9375-915b955f2301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2edeae76227b4d72acff2f4434e2cd8b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:40.752052 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b04dd28-270a-4fd5-9375-915b955f2301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2edeae76227b4d72acff2f4434e2cd8b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:40.753243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5b04dd28-270a-4fd5-9375-915b955f2301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2edeae76227b4d72acff2f4434e2cd8b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:40.767230 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1305/2598] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:40 2022] DELETE /identity/v3/users/2edeae76227b4d72acff2f4434e2cd8b => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:40.771879 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d74323-1ee3-4ce1-8528-a8256c71d403 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:40.772933 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d74323-1ee3-4ce1-8528-a8256c71d403 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:40.773510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d74323-1ee3-4ce1-8528-a8256c71d403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:40.817191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-11d74323-1ee3-4ce1-8528-a8256c71d403 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:40.825030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-11d74323-1ee3-4ce1-8528-a8256c71d403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:40.991079 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1294/2599] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:34:41.008330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04dd28-270a-4fd5-9375-915b955f2301 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:41.009904 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5b04dd28-270a-4fd5-9375-915b955f2301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:41.023963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:41.167085 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04dd28-270a-4fd5-9375-915b955f2301 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:41.172490 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360ab87-c83b-42e0-896f-44245ebc2e15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:41.173366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360ab87-c83b-42e0-896f-44245ebc2e15 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:41.173799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360ab87-c83b-42e0-896f-44245ebc2e15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:41.174225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360ab87-c83b-42e0-896f-44245ebc2e15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:41.192197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3360ab87-c83b-42e0-896f-44245ebc2e15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:41.194221 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3360ab87-c83b-42e0-896f-44245ebc2e15 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:41.206178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1306/2600] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:41.836631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d74323-1ee3-4ce1-8528-a8256c71d403 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:41.838419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-11d74323-1ee3-4ce1-8528-a8256c71d403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:41.855020 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d74323-1ee3-4ce1-8528-a8256c71d403 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:41.865550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:41.867355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:41.868760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:41.869870 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] PATH_INFO: `/v3/projects/959990a759224e47b146386f4f939a79` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:41.871807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=959990a759224e47b146386f4f939a79)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:41.874819 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:41.967152 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '959990a759224e47b146386f4f939a79', 'request_id': 'req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:41.975965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '959990a759224e47b146386f4f939a79', 'request_id': 'req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:41.999540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '959990a759224e47b146386f4f939a79', 'request_id': 'req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:42.014429 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] The token cache is being invalidate because project 959990a759224e47b146386f4f939a79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:42.015108 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:42.016117 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:42.019010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1295/2601] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:41 2022] DELETE /identity/v3/projects/959990a759224e47b146386f4f939a79 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:42.023739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360ab87-c83b-42e0-896f-44245ebc2e15 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:42.035073 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360ab87-c83b-42e0-896f-44245ebc2e15 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:42.144741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a56a6a-5253-4149-b44d-d11eb5c09737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:42.145432 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a56a6a-5253-4149-b44d-d11eb5c09737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:42.145719 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a56a6a-5253-4149-b44d-d11eb5c09737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:42.145995 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a56a6a-5253-4149-b44d-d11eb5c09737 admin admin] PATH_INFO: `/v3/users/437dc85e8ed6472fa4137780b43b70b6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:42.148555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a56a6a-5253-4149-b44d-d11eb5c09737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=437dc85e8ed6472fa4137780b43b70b6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:42.156950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a56a6a-5253-4149-b44d-d11eb5c09737 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:42.265162 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d9a56a6a-5253-4149-b44d-d11eb5c09737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '437dc85e8ed6472fa4137780b43b70b6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:42.275753 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d9a56a6a-5253-4149-b44d-d11eb5c09737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '437dc85e8ed6472fa4137780b43b70b6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:42.292718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d9a56a6a-5253-4149-b44d-d11eb5c09737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '437dc85e8ed6472fa4137780b43b70b6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:42.305206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d9a56a6a-5253-4149-b44d-d11eb5c09737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '437dc85e8ed6472fa4137780b43b70b6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:42.306469 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d9a56a6a-5253-4149-b44d-d11eb5c09737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '437dc85e8ed6472fa4137780b43b70b6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:42.318909 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1307/2602] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:42 2022] DELETE /identity/v3/users/437dc85e8ed6472fa4137780b43b70b6 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:42.774796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:42.776902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:42.795428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359b3f6-cfb3-4e9c-bd8f-1b936d9302b0 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:42.878394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133740a-5e30-45ff-b305-18db86b073e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:42.879207 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133740a-5e30-45ff-b305-18db86b073e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:42.883614 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133740a-5e30-45ff-b305-18db86b073e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:42.884161 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133740a-5e30-45ff-b305-18db86b073e4 admin admin] PATH_INFO: `/v3/projects/d0ce4d30056f48d1bccc59d8f894248b` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:42.885561 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133740a-5e30-45ff-b305-18db86b073e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ce4d30056f48d1bccc59d8f894248b)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:42.887646 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133740a-5e30-45ff-b305-18db86b073e4 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:42.906263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:42.970817 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7133740a-5e30-45ff-b305-18db86b073e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ce4d30056f48d1bccc59d8f894248b', 'request_id': 'req-7133740a-5e30-45ff-b305-18db86b073e4'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:42.978729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7133740a-5e30-45ff-b305-18db86b073e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ce4d30056f48d1bccc59d8f894248b', 'request_id': 'req-7133740a-5e30-45ff-b305-18db86b073e4'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:42.988487 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7133740a-5e30-45ff-b305-18db86b073e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ce4d30056f48d1bccc59d8f894248b', 'request_id': 'req-7133740a-5e30-45ff-b305-18db86b073e4'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:43.014542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7133740a-5e30-45ff-b305-18db86b073e4 admin admin] The token cache is being invalidate because project d0ce4d30056f48d1bccc59d8f894248b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:43.014542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7133740a-5e30-45ff-b305-18db86b073e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:43.015330 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7133740a-5e30-45ff-b305-18db86b073e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:43.017904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1296/2603] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:42 2022] DELETE /identity/v3/projects/d0ce4d30056f48d1bccc59d8f894248b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:43.023153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150814e-5e14-48cf-a7ac-da851a50dfac None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:43.024408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150814e-5e14-48cf-a7ac-da851a50dfac None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:43.025335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150814e-5e14-48cf-a7ac-da851a50dfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:43.063597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4150814e-5e14-48cf-a7ac-da851a50dfac None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:43.066695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4150814e-5e14-48cf-a7ac-da851a50dfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:43.145931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:43.219099 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1308/2604] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:43.227213 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909d4c9-fea8-4903-9ee3-904818f9abe5 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:43.227510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909d4c9-fea8-4903-9ee3-904818f9abe5 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:43.227813 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909d4c9-fea8-4903-9ee3-904818f9abe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:43.261862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-4909d4c9-fea8-4903-9ee3-904818f9abe5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:43.263462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4909d4c9-fea8-4903-9ee3-904818f9abe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:43.275969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1297/2605] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:43.281126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150814e-5e14-48cf-a7ac-da851a50dfac None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:43.296937 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150814e-5e14-48cf-a7ac-da851a50dfac None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:43.302499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f024a-7945-4042-b316-a030e30af655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:43.303207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f024a-7945-4042-b316-a030e30af655 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:43.303561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f024a-7945-4042-b316-a030e30af655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:43.304005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f024a-7945-4042-b316-a030e30af655 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:43.304747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f024a-7945-4042-b316-a030e30af655 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:34:43.305142 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f024a-7945-4042-b316-a030e30af655 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:43.306848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f024a-7945-4042-b316-a030e30af655 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:43.323628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1309/2606] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:34:43.325563 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909d4c9-fea8-4903-9ee3-904818f9abe5 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:43.335896 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909d4c9-fea8-4903-9ee3-904818f9abe5 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:43.340936 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969574a-3dbc-40b0-ba44-a46a643e4421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:43.341554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969574a-3dbc-40b0-ba44-a46a643e4421 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:43.341832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969574a-3dbc-40b0-ba44-a46a643e4421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:43.342103 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969574a-3dbc-40b0-ba44-a46a643e4421 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:43.342746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969574a-3dbc-40b0-ba44-a46a643e4421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:43.344773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969574a-3dbc-40b0-ba44-a46a643e4421 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:43.382010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1298/2607] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:43 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:43.383536 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f024a-7945-4042-b316-a030e30af655 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:43.395691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f024a-7945-4042-b316-a030e30af655 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:43.460739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89377d94-5cd6-4dc4-9968-4cafbc4799ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:43.461467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89377d94-5cd6-4dc4-9968-4cafbc4799ee admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:43.461806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89377d94-5cd6-4dc4-9968-4cafbc4799ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:43.462186 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89377d94-5cd6-4dc4-9968-4cafbc4799ee admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:43.462833 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89377d94-5cd6-4dc4-9968-4cafbc4799ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:43.464879 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89377d94-5cd6-4dc4-9968-4cafbc4799ee admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:43.477665 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b2a97-cc59-4dbd-8b93-fa3fecfb17d9 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:43.478028 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b2a97-cc59-4dbd-8b93-fa3fecfb17d9 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:43.478308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b2a97-cc59-4dbd-8b93-fa3fecfb17d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:43.489330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1310/2608] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:43 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:43.495145 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89377d94-5cd6-4dc4-9968-4cafbc4799ee admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:43.506304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89377d94-5cd6-4dc4-9968-4cafbc4799ee admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:43.509437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-cf0b2a97-cc59-4dbd-8b93-fa3fecfb17d9 None None] MFA Rules not processed for user `d587495071784b1da16a0d771b5bfc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:43.510691 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-cf0b2a97-cc59-4dbd-8b93-fa3fecfb17d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:43.511245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8d563-b9d5-467c-96c8-f039e514a6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:43.511888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8d563-b9d5-467c-96c8-f039e514a6af admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:43.512173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8d563-b9d5-467c-96c8-f039e514a6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:43.512449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8d563-b9d5-467c-96c8-f039e514a6af admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:43.513089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8d563-b9d5-467c-96c8-f039e514a6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:43.514775 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8d563-b9d5-467c-96c8-f039e514a6af admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:43.527239 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1311/2609] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:43.531214 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8d563-b9d5-467c-96c8-f039e514a6af admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:43.541890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8d563-b9d5-467c-96c8-f039e514a6af admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:43.546959 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9828910-35a0-43d3-90b5-d389d78af659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:43.547608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9828910-35a0-43d3-90b5-d389d78af659 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:43.547888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9828910-35a0-43d3-90b5-d389d78af659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:43.548171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9828910-35a0-43d3-90b5-d389d78af659 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:43.548819 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9828910-35a0-43d3-90b5-d389d78af659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:43.550570 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9828910-35a0-43d3-90b5-d389d78af659 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:43.568587 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1312/2610] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:43.570105 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9828910-35a0-43d3-90b5-d389d78af659 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:43.583001 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9828910-35a0-43d3-90b5-d389d78af659 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:43.589185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ea0d2-72c4-4458-aea4-26dd0e67e3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:43.590156 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ea0d2-72c4-4458-aea4-26dd0e67e3bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:43.590449 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ea0d2-72c4-4458-aea4-26dd0e67e3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:43.590730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ea0d2-72c4-4458-aea4-26dd0e67e3bf admin admin] PATH_INFO: `/v3/projects/d7f9639368ad44e5907c23f9f9170b2b/users/98da5f5462034be9ac244000e70901af/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:43.593008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1299/2611] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:34:43.606267 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b2a97-cc59-4dbd-8b93-fa3fecfb17d9 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:43.607752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-cf0b2a97-cc59-4dbd-8b93-fa3fecfb17d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:43.611039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ea0d2-72c4-4458-aea4-26dd0e67e3bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f9639368ad44e5907c23f9f9170b2b, user_id=98da5f5462034be9ac244000e70901af, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:43.615061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ea0d2-72c4-4458-aea4-26dd0e67e3bf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:43.632601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1313/2612] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:43 2022] PUT /identity/v3/projects/d7f9639368ad44e5907c23f9f9170b2b/users/98da5f5462034be9ac244000e70901af/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:43.644751 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72c060-4964-42d4-879c-0d9a3ba15cff None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:43.645043 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72c060-4964-42d4-879c-0d9a3ba15cff None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:43.645350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72c060-4964-42d4-879c-0d9a3ba15cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:43.678725 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-da72c060-4964-42d4-879c-0d9a3ba15cff None None] MFA Rules not processed for user `98da5f5462034be9ac244000e70901af`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:43.680150 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-da72c060-4964-42d4-879c-0d9a3ba15cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:43.689780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b2a97-cc59-4dbd-8b93-fa3fecfb17d9 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:43.707194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:43.745977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:43.758653 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a30dc4-c865-44d8-9e78-8be4f4320115 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61407406ef242a78030f79532fe2247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:43.759269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a30dc4-c865-44d8-9e78-8be4f4320115 service glance] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:43.759549 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a30dc4-c865-44d8-9e78-8be4f4320115 service glance] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:43.759973 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a30dc4-c865-44d8-9e78-8be4f4320115 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:43.762625 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d4a30dc4-c865-44d8-9e78-8be4f4320115 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:43.775950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1314/2613] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:34:43.779488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72c060-4964-42d4-879c-0d9a3ba15cff None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:43.852215 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a30dc4-c865-44d8-9e78-8be4f4320115 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:43.853447 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a30dc4-c865-44d8-9e78-8be4f4320115 service glance] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:43.865188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1300/2614] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:43.871884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72c060-4964-42d4-879c-0d9a3ba15cff None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:43.877156 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffad0b6-f64b-48dd-8735-7df47829764c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:43.877767 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffad0b6-f64b-48dd-8735-7df47829764c admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:43.878048 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffad0b6-f64b-48dd-8735-7df47829764c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:43.878351 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffad0b6-f64b-48dd-8735-7df47829764c admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:43.878997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffad0b6-f64b-48dd-8735-7df47829764c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:43.881586 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffad0b6-f64b-48dd-8735-7df47829764c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:43.915346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1315/2615] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:43 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:34:43.916772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a30dc4-c865-44d8-9e78-8be4f4320115 service glance] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:43.933657 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a30dc4-c865-44d8-9e78-8be4f4320115 service glance] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:44.008899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c325884-c8be-477e-8623-67afb62a2c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:44.009442 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c325884-c8be-477e-8623-67afb62a2c41 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:44.011314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c325884-c8be-477e-8623-67afb62a2c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:44.011314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c325884-c8be-477e-8623-67afb62a2c41 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:44.011314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c325884-c8be-477e-8623-67afb62a2c41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:44.012611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c325884-c8be-477e-8623-67afb62a2c41 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:44.014418 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64ac0-6217-418d-8b5a-a30a14c15dc9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:44.014734 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64ac0-6217-418d-8b5a-a30a14c15dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:44.015012 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64ac0-6217-418d-8b5a-a30a14c15dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:44.041851 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1301/2616] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:43 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:44.042066 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-efb64ac0-6217-418d-8b5a-a30a14c15dc9 None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:44.042066 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-efb64ac0-6217-418d-8b5a-a30a14c15dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:44.042954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c325884-c8be-477e-8623-67afb62a2c41 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:44.052667 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c325884-c8be-477e-8623-67afb62a2c41 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:44.057453 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112d0c2-8247-43e0-9981-1d0010f355af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:44.058061 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112d0c2-8247-43e0-9981-1d0010f355af admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:44.058332 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112d0c2-8247-43e0-9981-1d0010f355af admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:44.058611 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112d0c2-8247-43e0-9981-1d0010f355af admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:44.059254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112d0c2-8247-43e0-9981-1d0010f355af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:44.060902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112d0c2-8247-43e0-9981-1d0010f355af admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:44.071189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1302/2617] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:44.075556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112d0c2-8247-43e0-9981-1d0010f355af admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:44.088212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112d0c2-8247-43e0-9981-1d0010f355af admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:44.093350 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219db9c-c39a-4c01-b810-9992bfb87d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:44.094226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219db9c-c39a-4c01-b810-9992bfb87d40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:44.094508 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219db9c-c39a-4c01-b810-9992bfb87d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:44.094787 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219db9c-c39a-4c01-b810-9992bfb87d40 admin admin] PATH_INFO: `/v3/projects/3bd77dc0f13f4fc0aa3b7441792f9d4a/users/d9ce7f13a68c431f91b7802cafdf2651/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:44.112541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219db9c-c39a-4c01-b810-9992bfb87d40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd77dc0f13f4fc0aa3b7441792f9d4a, user_id=d9ce7f13a68c431f91b7802cafdf2651, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:44.116366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219db9c-c39a-4c01-b810-9992bfb87d40 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:44.132170 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1303/2618] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:44 2022] PUT /identity/v3/projects/3bd77dc0f13f4fc0aa3b7441792f9d4a/users/d9ce7f13a68c431f91b7802cafdf2651/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:34:44.136340 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219db9c-c39a-4c01-b810-9992bfb87d40 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:44.169189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:44.205848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219db9c-c39a-4c01-b810-9992bfb87d40 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:44.210812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67834a69-efae-4c86-abf8-c34fd7bec9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:44.211423 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67834a69-efae-4c86-abf8-c34fd7bec9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:44.211733 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67834a69-efae-4c86-abf8-c34fd7bec9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:44.212003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67834a69-efae-4c86-abf8-c34fd7bec9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:44.212639 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67834a69-efae-4c86-abf8-c34fd7bec9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:44.214310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67834a69-efae-4c86-abf8-c34fd7bec9d3 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:44.224982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1304/2619] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:34:44.229000 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67834a69-efae-4c86-abf8-c34fd7bec9d3 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:44.233036 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1316/2620] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:34:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:44.238796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67834a69-efae-4c86-abf8-c34fd7bec9d3 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:44.249719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b75245-be41-4222-a586-2f457844acc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:44.249719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b75245-be41-4222-a586-2f457844acc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:44.249719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b75245-be41-4222-a586-2f457844acc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:44.249719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b75245-be41-4222-a586-2f457844acc1 admin admin] PATH_INFO: `/v3/projects/3bd77dc0f13f4fc0aa3b7441792f9d4a/users/d9ce7f13a68c431f91b7802cafdf2651/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:44.250148 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b75245-be41-4222-a586-2f457844acc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd77dc0f13f4fc0aa3b7441792f9d4a, user_id=d9ce7f13a68c431f91b7802cafdf2651, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:44.250641 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b75245-be41-4222-a586-2f457844acc1 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:44.259171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb64ac0-6217-418d-8b5a-a30a14c15dc9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:44.260600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-efb64ac0-6217-418d-8b5a-a30a14c15dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:44.264972 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1305/2621] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:44 2022] PUT /identity/v3/projects/3bd77dc0f13f4fc0aa3b7441792f9d4a/users/d9ce7f13a68c431f91b7802cafdf2651/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:44.276727 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2efb0c-e9c5-4e5c-948c-96d8cffa6e42 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:44.277009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2efb0c-e9c5-4e5c-948c-96d8cffa6e42 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:44.277281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2efb0c-e9c5-4e5c-948c-96d8cffa6e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:44.314975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-bb2efb0c-e9c5-4e5c-948c-96d8cffa6e42 None None] MFA Rules not processed for user `d9ce7f13a68c431f91b7802cafdf2651`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:44.316454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bb2efb0c-e9c5-4e5c-948c-96d8cffa6e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:44.351087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb64ac0-6217-418d-8b5a-a30a14c15dc9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:44.372165 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:44.402931 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1306/2622] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:34:44.415113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2efb0c-e9c5-4e5c-948c-96d8cffa6e42 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:44.416583 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bb2efb0c-e9c5-4e5c-948c-96d8cffa6e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:44.422445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430e555-4beb-4248-8148-d8a6f4438af1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:44.423237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430e555-4beb-4248-8148-d8a6f4438af1 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:44.424154 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430e555-4beb-4248-8148-d8a6f4438af1 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:44.424840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430e555-4beb-4248-8148-d8a6f4438af1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:44.503673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430e555-4beb-4248-8148-d8a6f4438af1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:44.505993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430e555-4beb-4248-8148-d8a6f4438af1 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:44.515316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1317/2623] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:34:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 23:34:44.521161 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2efb0c-e9c5-4e5c-948c-96d8cffa6e42 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:44.525281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370a01f-8dc2-4c81-bb3f-8fdc9a9f1f7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:44.525949 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370a01f-8dc2-4c81-bb3f-8fdc9a9f1f7b service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:44.526264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370a01f-8dc2-4c81-bb3f-8fdc9a9f1f7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:44.526540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370a01f-8dc2-4c81-bb3f-8fdc9a9f1f7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:44.539243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370a01f-8dc2-4c81-bb3f-8fdc9a9f1f7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:44.540917 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370a01f-8dc2-4c81-bb3f-8fdc9a9f1f7b service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:44.572868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1307/2624] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:34:44.657314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430e555-4beb-4248-8148-d8a6f4438af1 service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:44.761597 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430e555-4beb-4248-8148-d8a6f4438af1 service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:44.768538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ad887-4f93-4b19-bf25-7e71761ca428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:44.769189 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ad887-4f93-4b19-bf25-7e71761ca428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:44.769472 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ad887-4f93-4b19-bf25-7e71761ca428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:44.769746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ad887-4f93-4b19-bf25-7e71761ca428 admin admin] PATH_INFO: `/v3/users/98da5f5462034be9ac244000e70901af` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:44.772217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ad887-4f93-4b19-bf25-7e71761ca428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98da5f5462034be9ac244000e70901af)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:44.774414 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ad887-4f93-4b19-bf25-7e71761ca428 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:44.848951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f11ad887-4f93-4b19-bf25-7e71761ca428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98da5f5462034be9ac244000e70901af'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:44.857755 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f11ad887-4f93-4b19-bf25-7e71761ca428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98da5f5462034be9ac244000e70901af'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:44.868092 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:44.878023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f11ad887-4f93-4b19-bf25-7e71761ca428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98da5f5462034be9ac244000e70901af'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:44.891649 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f11ad887-4f93-4b19-bf25-7e71761ca428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98da5f5462034be9ac244000e70901af'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:44.898003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f11ad887-4f93-4b19-bf25-7e71761ca428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98da5f5462034be9ac244000e70901af'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:44.909785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1318/2625] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:44 2022] DELETE /identity/v3/users/98da5f5462034be9ac244000e70901af => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:44.916287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632deb2-b5ce-4982-8b13-d1f7490ba51c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:44.916287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632deb2-b5ce-4982-8b13-d1f7490ba51c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:44.916602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632deb2-b5ce-4982-8b13-d1f7490ba51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:44.949326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-2632deb2-b5ce-4982-8b13-d1f7490ba51c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:44.950859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2632deb2-b5ce-4982-8b13-d1f7490ba51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:44.992130 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:45.043543 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1308/2626] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:45.050882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ad887-4f93-4b19-bf25-7e71761ca428 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:45.053979 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f11ad887-4f93-4b19-bf25-7e71761ca428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:45.129862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ad887-4f93-4b19-bf25-7e71761ca428 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:45.133600 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517834a5-d10f-4cbb-98dd-d245e10128cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:45.134212 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517834a5-d10f-4cbb-98dd-d245e10128cc service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:45.134495 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517834a5-d10f-4cbb-98dd-d245e10128cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:45.134772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517834a5-d10f-4cbb-98dd-d245e10128cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:45.146876 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517834a5-d10f-4cbb-98dd-d245e10128cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:45.153497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517834a5-d10f-4cbb-98dd-d245e10128cc service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:45.167982 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1319/2627] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:34:45.532904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632deb2-b5ce-4982-8b13-d1f7490ba51c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:45.534729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2632deb2-b5ce-4982-8b13-d1f7490ba51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:45.553180 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632deb2-b5ce-4982-8b13-d1f7490ba51c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:45.557898 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:45.558662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:45.558959 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:45.559273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] PATH_INFO: `/v3/projects/d7f9639368ad44e5907c23f9f9170b2b` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:45.560470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7f9639368ad44e5907c23f9f9170b2b)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:45.563210 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:45.635923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7f9639368ad44e5907c23f9f9170b2b', 'request_id': 'req-506ca892-8be7-445c-8102-fe728fc03e59'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:45.643115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7f9639368ad44e5907c23f9f9170b2b', 'request_id': 'req-506ca892-8be7-445c-8102-fe728fc03e59'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:45.654781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7f9639368ad44e5907c23f9f9170b2b', 'request_id': 'req-506ca892-8be7-445c-8102-fe728fc03e59'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:45.673349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] The token cache is being invalidate because project d7f9639368ad44e5907c23f9f9170b2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:45.673794 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:45.675246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:45.677863 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1309/2628] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:45 2022] DELETE /identity/v3/projects/d7f9639368ad44e5907c23f9f9170b2b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:45.681445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517834a5-d10f-4cbb-98dd-d245e10128cc service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:45.693864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517834a5-d10f-4cbb-98dd-d245e10128cc service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:45.772557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153831f4-b9cf-4285-b1dc-059465a1a35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:45.773505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153831f4-b9cf-4285-b1dc-059465a1a35c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:45.773934 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153831f4-b9cf-4285-b1dc-059465a1a35c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:45.774635 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153831f4-b9cf-4285-b1dc-059465a1a35c admin admin] PATH_INFO: `/v3/users/d9ce7f13a68c431f91b7802cafdf2651` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:45.775981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153831f4-b9cf-4285-b1dc-059465a1a35c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9ce7f13a68c431f91b7802cafdf2651)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:45.778111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153831f4-b9cf-4285-b1dc-059465a1a35c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:45.842974 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-153831f4-b9cf-4285-b1dc-059465a1a35c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9ce7f13a68c431f91b7802cafdf2651'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:45.854242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-153831f4-b9cf-4285-b1dc-059465a1a35c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9ce7f13a68c431f91b7802cafdf2651'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:45.936644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-153831f4-b9cf-4285-b1dc-059465a1a35c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9ce7f13a68c431f91b7802cafdf2651'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:45.950502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-153831f4-b9cf-4285-b1dc-059465a1a35c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9ce7f13a68c431f91b7802cafdf2651'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:45.951452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-153831f4-b9cf-4285-b1dc-059465a1a35c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9ce7f13a68c431f91b7802cafdf2651'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:45.963604 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1320/2629] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:45 2022] DELETE /identity/v3/users/d9ce7f13a68c431f91b7802cafdf2651 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:34:46.325076 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:46.326977 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:46.361454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ca892-8be7-445c-8102-fe728fc03e59 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:46.453407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:46.453407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:46.453407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:46.453407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5 admin admin] PATH_INFO: `/v3/projects/3bd77dc0f13f4fc0aa3b7441792f9d4a` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:46.453924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd77dc0f13f4fc0aa3b7441792f9d4a)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:46.454777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:46.517844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd77dc0f13f4fc0aa3b7441792f9d4a', 'request_id': 'req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:46.525179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd77dc0f13f4fc0aa3b7441792f9d4a', 'request_id': 'req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:46.529752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:46.539101 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd77dc0f13f4fc0aa3b7441792f9d4a', 'request_id': 'req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:46.555789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5 admin admin] The token cache is being invalidate because project 3bd77dc0f13f4fc0aa3b7441792f9d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:46.556429 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:46.557388 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-18c8dc27-ba90-4b6b-b3dc-a496cc5c8fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:46.563654 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1310/2630] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:46 2022] DELETE /identity/v3/projects/3bd77dc0f13f4fc0aa3b7441792f9d4a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:46.565169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe998df0-6613-463a-bca3-cafe439ba741 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:46.565638 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe998df0-6613-463a-bca3-cafe439ba741 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:46.565938 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe998df0-6613-463a-bca3-cafe439ba741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:46.577429 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:46.600020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-fe998df0-6613-463a-bca3-cafe439ba741 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:46.601350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fe998df0-6613-463a-bca3-cafe439ba741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:46.757679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1321/2631] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:46.760736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7c0ab-8e9e-496f-95fe-5908e2b86620 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:46.760736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7c0ab-8e9e-496f-95fe-5908e2b86620 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:46.791257 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7c0ab-8e9e-496f-95fe-5908e2b86620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:46.820774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-aae7c0ab-8e9e-496f-95fe-5908e2b86620 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:46.822030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-aae7c0ab-8e9e-496f-95fe-5908e2b86620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:46.830736 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1311/2632] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:46.835689 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe998df0-6613-463a-bca3-cafe439ba741 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:46.852578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe998df0-6613-463a-bca3-cafe439ba741 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:46.857697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fe2ff-83de-449b-985f-e6604e2cdb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:46.858406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fe2ff-83de-449b-985f-e6604e2cdb19 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:46.858706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fe2ff-83de-449b-985f-e6604e2cdb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:46.858989 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fe2ff-83de-449b-985f-e6604e2cdb19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:46.859765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110fe2ff-83de-449b-985f-e6604e2cdb19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:34:46.860073 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110fe2ff-83de-449b-985f-e6604e2cdb19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:46.861908 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110fe2ff-83de-449b-985f-e6604e2cdb19 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:46.875652 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1322/2633] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:34:46.879808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7c0ab-8e9e-496f-95fe-5908e2b86620 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:46.894226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7c0ab-8e9e-496f-95fe-5908e2b86620 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:46.900903 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1cfbb-7dc7-4843-868e-bb04d987c34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:46.901727 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1cfbb-7dc7-4843-868e-bb04d987c34b admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:46.902108 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1cfbb-7dc7-4843-868e-bb04d987c34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:46.902479 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1cfbb-7dc7-4843-868e-bb04d987c34b admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:46.903215 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1cfbb-7dc7-4843-868e-bb04d987c34b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:46.905499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1cfbb-7dc7-4843-868e-bb04d987c34b admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:46.954884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1312/2634] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:46 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:46.959488 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fe2ff-83de-449b-985f-e6604e2cdb19 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:46.970625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fe2ff-83de-449b-985f-e6604e2cdb19 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:47.055609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ed08e-d794-4d7c-b3c9-35840a261644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:47.056785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ed08e-d794-4d7c-b3c9-35840a261644 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:47.057222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ed08e-d794-4d7c-b3c9-35840a261644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:47.057648 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ed08e-d794-4d7c-b3c9-35840a261644 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:47.058489 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ed08e-d794-4d7c-b3c9-35840a261644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:47.060625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ed08e-d794-4d7c-b3c9-35840a261644 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:47.092133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1323/2635] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:46 2022] POST /identity/v3/users => generated 290 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:34:47.096375 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1cfbb-7dc7-4843-868e-bb04d987c34b admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:47.107545 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1cfbb-7dc7-4843-868e-bb04d987c34b admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:47.113199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b571be-1120-4c76-a87f-aa957a600074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:47.114439 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b571be-1120-4c76-a87f-aa957a600074 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:47.114791 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b571be-1120-4c76-a87f-aa957a600074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:47.115134 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b571be-1120-4c76-a87f-aa957a600074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:47.115897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b571be-1120-4c76-a87f-aa957a600074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:47.117662 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b571be-1120-4c76-a87f-aa957a600074 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:47.130644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1313/2636] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:47.134794 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ed08e-d794-4d7c-b3c9-35840a261644 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:47.145656 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ed08e-d794-4d7c-b3c9-35840a261644 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:47.154108 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b313a4-5b82-4aad-9417-0b68acba44ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:47.155112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b313a4-5b82-4aad-9417-0b68acba44ad admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:47.155765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b313a4-5b82-4aad-9417-0b68acba44ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:47.156237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b313a4-5b82-4aad-9417-0b68acba44ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:47.157100 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b313a4-5b82-4aad-9417-0b68acba44ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:47.159228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b313a4-5b82-4aad-9417-0b68acba44ad admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:47.172763 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1324/2637] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:47.178069 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b571be-1120-4c76-a87f-aa957a600074 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:47.188609 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b571be-1120-4c76-a87f-aa957a600074 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:47.193825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45764ffc-b0f9-4f1b-af84-a61034131bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:47.194599 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45764ffc-b0f9-4f1b-af84-a61034131bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:47.194977 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45764ffc-b0f9-4f1b-af84-a61034131bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:47.195349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45764ffc-b0f9-4f1b-af84-a61034131bc5 admin admin] PATH_INFO: `/v3/projects/67131ac142914dd4a311c2a30ef4228d/users/a503387af43b492aa1896561f85a52ed/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:47.221083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45764ffc-b0f9-4f1b-af84-a61034131bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67131ac142914dd4a311c2a30ef4228d, user_id=a503387af43b492aa1896561f85a52ed, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:47.224397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45764ffc-b0f9-4f1b-af84-a61034131bc5 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:47.244841 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1314/2638] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:47 2022] PUT /identity/v3/projects/67131ac142914dd4a311c2a30ef4228d/users/a503387af43b492aa1896561f85a52ed/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:34:47.257036 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abd327-8512-4f3b-b539-a033295bdbdf None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:47.257350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abd327-8512-4f3b-b539-a033295bdbdf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:47.257635 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abd327-8512-4f3b-b539-a033295bdbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:47.298876 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-20abd327-8512-4f3b-b539-a033295bdbdf None None] MFA Rules not processed for user `a503387af43b492aa1896561f85a52ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:47.300256 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-20abd327-8512-4f3b-b539-a033295bdbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:47.381704 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:47.391117 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1325/2639] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:47.392003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45764ffc-b0f9-4f1b-af84-a61034131bc5 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:47.464025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45764ffc-b0f9-4f1b-af84-a61034131bc5 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:47.469028 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7266d5-44c6-4e30-b832-fbc2b23ff1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:47.469635 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7266d5-44c6-4e30-b832-fbc2b23ff1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:47.469916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7266d5-44c6-4e30-b832-fbc2b23ff1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:47.470190 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7266d5-44c6-4e30-b832-fbc2b23ff1c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:47.470853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7266d5-44c6-4e30-b832-fbc2b23ff1c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:47.472817 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7266d5-44c6-4e30-b832-fbc2b23ff1c1 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:47.511205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1315/2640] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:47 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:34:47.512400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abd327-8512-4f3b-b539-a033295bdbdf None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:47.531519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abd327-8512-4f3b-b539-a033295bdbdf None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:47.598881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249a1bd-6038-4e1a-a0f5-466cadbb1fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:47.599621 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249a1bd-6038-4e1a-a0f5-466cadbb1fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:47.599780 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249a1bd-6038-4e1a-a0f5-466cadbb1fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:47.600051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249a1bd-6038-4e1a-a0f5-466cadbb1fe5 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:47.600710 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5249a1bd-6038-4e1a-a0f5-466cadbb1fe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:47.602612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5249a1bd-6038-4e1a-a0f5-466cadbb1fe5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:47.626414 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1326/2641] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:47 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:47.630541 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7266d5-44c6-4e30-b832-fbc2b23ff1c1 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:47.644040 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7266d5-44c6-4e30-b832-fbc2b23ff1c1 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:47.649797 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf321153-b236-4e55-99e7-1a6ec6be49f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:47.650517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf321153-b236-4e55-99e7-1a6ec6be49f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:47.650897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf321153-b236-4e55-99e7-1a6ec6be49f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:47.651271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf321153-b236-4e55-99e7-1a6ec6be49f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:47.653373 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf321153-b236-4e55-99e7-1a6ec6be49f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:47.654097 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf321153-b236-4e55-99e7-1a6ec6be49f1 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:47.667675 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1316/2642] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:47.671935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249a1bd-6038-4e1a-a0f5-466cadbb1fe5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:47.692569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249a1bd-6038-4e1a-a0f5-466cadbb1fe5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:47.700515 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e6480-02bf-43f0-ae4c-60df27b78597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:47.701205 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e6480-02bf-43f0-ae4c-60df27b78597 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:47.702501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e6480-02bf-43f0-ae4c-60df27b78597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:47.702501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e6480-02bf-43f0-ae4c-60df27b78597 admin admin] PATH_INFO: `/v3/projects/28497684f6cb4f2fb019d1993a85b1b3/users/b954bb76ed5041e6801a10f3a9dcb060/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:47.724878 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e6480-02bf-43f0-ae4c-60df27b78597 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28497684f6cb4f2fb019d1993a85b1b3, user_id=b954bb76ed5041e6801a10f3a9dcb060, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:47.728204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e6480-02bf-43f0-ae4c-60df27b78597 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:47.747625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1327/2643] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:47 2022] PUT /identity/v3/projects/28497684f6cb4f2fb019d1993a85b1b3/users/b954bb76ed5041e6801a10f3a9dcb060/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:47.750409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf321153-b236-4e55-99e7-1a6ec6be49f1 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:47.762427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf321153-b236-4e55-99e7-1a6ec6be49f1 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:47.830586 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681a3ef-17c3-473f-b52b-affc3fd0ad21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:47.835363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681a3ef-17c3-473f-b52b-affc3fd0ad21 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:47.835363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681a3ef-17c3-473f-b52b-affc3fd0ad21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:47.835363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681a3ef-17c3-473f-b52b-affc3fd0ad21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:47.835363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681a3ef-17c3-473f-b52b-affc3fd0ad21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:47.835363 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681a3ef-17c3-473f-b52b-affc3fd0ad21 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:47.837912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:47.854627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1317/2644] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:47.856802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e6480-02bf-43f0-ae4c-60df27b78597 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:47.867813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e6480-02bf-43f0-ae4c-60df27b78597 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:47.873198 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5135376-c770-4f07-b5b1-801fa83e770d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:47.874029 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5135376-c770-4f07-b5b1-801fa83e770d admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:47.874309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5135376-c770-4f07-b5b1-801fa83e770d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:47.874581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5135376-c770-4f07-b5b1-801fa83e770d admin admin] PATH_INFO: `/v3/projects/28497684f6cb4f2fb019d1993a85b1b3/users/b954bb76ed5041e6801a10f3a9dcb060/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:47.877072 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5135376-c770-4f07-b5b1-801fa83e770d admin admin] RBAC: Authorizing `identity:create_grant(project_id=28497684f6cb4f2fb019d1993a85b1b3, user_id=b954bb76ed5041e6801a10f3a9dcb060, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:47.880828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5135376-c770-4f07-b5b1-801fa83e770d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:47.897463 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1328/2645] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:47 2022] PUT /identity/v3/projects/28497684f6cb4f2fb019d1993a85b1b3/users/b954bb76ed5041e6801a10f3a9dcb060/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:47.915696 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28adae3-de92-4c2f-8f02-e74b31804357 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:47.916055 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28adae3-de92-4c2f-8f02-e74b31804357 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:47.916269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28adae3-de92-4c2f-8f02-e74b31804357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:47.958261 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c28adae3-de92-4c2f-8f02-e74b31804357 None None] MFA Rules not processed for user `b954bb76ed5041e6801a10f3a9dcb060`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:47.959916 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c28adae3-de92-4c2f-8f02-e74b31804357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:48.039515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1318/2646] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:48.052213 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5135376-c770-4f07-b5b1-801fa83e770d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:48.056771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d5135376-c770-4f07-b5b1-801fa83e770d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:48.129072 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5135376-c770-4f07-b5b1-801fa83e770d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:48.134245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b046a71-289c-474d-9ccb-603572b39041 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:48.134865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b046a71-289c-474d-9ccb-603572b39041 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:48.135145 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b046a71-289c-474d-9ccb-603572b39041 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:48.135411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b046a71-289c-474d-9ccb-603572b39041 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:48.146227 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b046a71-289c-474d-9ccb-603572b39041 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:48.147681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b046a71-289c-474d-9ccb-603572b39041 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:48.158467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1329/2647] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:48.644036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28adae3-de92-4c2f-8f02-e74b31804357 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:48.656366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28adae3-de92-4c2f-8f02-e74b31804357 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:48.661967 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e21576-8615-4ae7-ab13-ddabb884b3b8 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b954bb76ed5041e6801a10f3a9dcb060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28497684f6cb4f2fb019d1993a85b1b3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:48.662705 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e21576-8615-4ae7-ab13-ddabb884b3b8 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:48.663490 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e21576-8615-4ae7-ab13-ddabb884b3b8 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:48.663899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e21576-8615-4ae7-ab13-ddabb884b3b8 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:48.664665 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e21576-8615-4ae7-ab13-ddabb884b3b8 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:34:48.665113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e21576-8615-4ae7-ab13-ddabb884b3b8 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:48.667000 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e21576-8615-4ae7-ab13-ddabb884b3b8 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:48.691355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1319/2648] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:34:48.693111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b046a71-289c-474d-9ccb-603572b39041 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:48.709229 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b046a71-289c-474d-9ccb-603572b39041 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:48.721370 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2c3df-7e3d-4ffc-ac0f-e9f72f945dc1 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b954bb76ed5041e6801a10f3a9dcb060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28497684f6cb4f2fb019d1993a85b1b3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:48.722055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2c3df-7e3d-4ffc-ac0f-e9f72f945dc1 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:48.722346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2c3df-7e3d-4ffc-ac0f-e9f72f945dc1 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:48.722838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2c3df-7e3d-4ffc-ac0f-e9f72f945dc1 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:48.724110 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2c3df-7e3d-4ffc-ac0f-e9f72f945dc1 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:48.726293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2c3df-7e3d-4ffc-ac0f-e9f72f945dc1 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:48.772438 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1330/2649] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:48 2022] POST /identity/v3/projects => generated 353 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:48.777166 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e21576-8615-4ae7-ab13-ddabb884b3b8 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:48.789919 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e21576-8615-4ae7-ab13-ddabb884b3b8 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:48.881422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9c374-5416-40ae-817a-45d1c7e96a3b tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b954bb76ed5041e6801a10f3a9dcb060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28497684f6cb4f2fb019d1993a85b1b3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:48.882057 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9c374-5416-40ae-817a-45d1c7e96a3b tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:48.882338 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9c374-5416-40ae-817a-45d1c7e96a3b tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:48.882613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9c374-5416-40ae-817a-45d1c7e96a3b tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:48.883263 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9c374-5416-40ae-817a-45d1c7e96a3b tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:34:48.883584 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9c374-5416-40ae-817a-45d1c7e96a3b tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:48.885211 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9c374-5416-40ae-817a-45d1c7e96a3b tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:48.901014 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1320/2650] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:34:48.950604 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2c3df-7e3d-4ffc-ac0f-e9f72f945dc1 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:48.962220 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2c3df-7e3d-4ffc-ac0f-e9f72f945dc1 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:48.967195 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8b572-770f-47fd-87b0-fd86e0dd9787 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b954bb76ed5041e6801a10f3a9dcb060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28497684f6cb4f2fb019d1993a85b1b3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:48.967886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8b572-770f-47fd-87b0-fd86e0dd9787 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:48.968227 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8b572-770f-47fd-87b0-fd86e0dd9787 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:48.968508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8b572-770f-47fd-87b0-fd86e0dd9787 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] PATH_INFO: `/v3/projects/56999fe99252446ba9e15801a758e3ab` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:48.969782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8b572-770f-47fd-87b0-fd86e0dd9787 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] RBAC: Authorizing `identity:delete_project(project_id=56999fe99252446ba9e15801a758e3ab)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:48.971703 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8b572-770f-47fd-87b0-fd86e0dd9787 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:49.029078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:49.060004 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-84b8b572-770f-47fd-87b0-fd86e0dd9787 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56999fe99252446ba9e15801a758e3ab', 'request_id': 'req-84b8b572-770f-47fd-87b0-fd86e0dd9787'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:49.068088 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-84b8b572-770f-47fd-87b0-fd86e0dd9787 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56999fe99252446ba9e15801a758e3ab', 'request_id': 'req-84b8b572-770f-47fd-87b0-fd86e0dd9787'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:49.082090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-84b8b572-770f-47fd-87b0-fd86e0dd9787 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56999fe99252446ba9e15801a758e3ab', 'request_id': 'req-84b8b572-770f-47fd-87b0-fd86e0dd9787'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:49.100327 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-84b8b572-770f-47fd-87b0-fd86e0dd9787 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] The token cache is being invalidate because project 56999fe99252446ba9e15801a758e3ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:49.100877 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-84b8b572-770f-47fd-87b0-fd86e0dd9787 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:49.101761 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-84b8b572-770f-47fd-87b0-fd86e0dd9787 tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:49.103867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1331/2651] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:48 2022] DELETE /identity/v3/projects/56999fe99252446ba9e15801a758e3ab => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:49.382511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9c374-5416-40ae-817a-45d1c7e96a3b tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:49.384472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-26f9c374-5416-40ae-817a-45d1c7e96a3b tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:49.497554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9c374-5416-40ae-817a-45d1c7e96a3b tempest-VolumeQuotaClassesTest-1897738934 tempest-VolumeQuotaClassesTest-1897738934-project-admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:49.510412 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:49.571115 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e92f5-c712-42a8-b8ed-312296ef10a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:49.572131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e92f5-c712-42a8-b8ed-312296ef10a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:49.572618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e92f5-c712-42a8-b8ed-312296ef10a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:49.573041 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e92f5-c712-42a8-b8ed-312296ef10a2 admin admin] PATH_INFO: `/v3/users/a503387af43b492aa1896561f85a52ed` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:49.574418 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e92f5-c712-42a8-b8ed-312296ef10a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a503387af43b492aa1896561f85a52ed)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:49.576555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e92f5-c712-42a8-b8ed-312296ef10a2 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:49.642435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-276e92f5-c712-42a8-b8ed-312296ef10a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a503387af43b492aa1896561f85a52ed'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:49.661836 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-276e92f5-c712-42a8-b8ed-312296ef10a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a503387af43b492aa1896561f85a52ed'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:49.685464 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-276e92f5-c712-42a8-b8ed-312296ef10a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a503387af43b492aa1896561f85a52ed'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:49.686110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-276e92f5-c712-42a8-b8ed-312296ef10a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a503387af43b492aa1896561f85a52ed'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:49.701220 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-276e92f5-c712-42a8-b8ed-312296ef10a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a503387af43b492aa1896561f85a52ed'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:49.709448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1321/2652] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:49 2022] DELETE /identity/v3/users/a503387af43b492aa1896561f85a52ed => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:49.713985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49963d7-9fba-4b2c-a5c0-80584960aea6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:49.714456 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49963d7-9fba-4b2c-a5c0-80584960aea6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:49.714718 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49963d7-9fba-4b2c-a5c0-80584960aea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:49.737262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a49963d7-9fba-4b2c-a5c0-80584960aea6 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:49.739859 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a49963d7-9fba-4b2c-a5c0-80584960aea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:49.817625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1332/2653] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:34:49.828299 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e92f5-c712-42a8-b8ed-312296ef10a2 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:49.829746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-276e92f5-c712-42a8-b8ed-312296ef10a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:49.895979 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e92f5-c712-42a8-b8ed-312296ef10a2 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:49.900702 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078b13f-3c4e-48e2-a823-3e9381dfb994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:49.901496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078b13f-3c4e-48e2-a823-3e9381dfb994 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:49.901925 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078b13f-3c4e-48e2-a823-3e9381dfb994 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:49.902371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078b13f-3c4e-48e2-a823-3e9381dfb994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:49.922369 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078b13f-3c4e-48e2-a823-3e9381dfb994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:49.924273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078b13f-3c4e-48e2-a823-3e9381dfb994 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:49.950205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1322/2654] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:50.248986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49963d7-9fba-4b2c-a5c0-80584960aea6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:50.250466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a49963d7-9fba-4b2c-a5c0-80584960aea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:50.270543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49963d7-9fba-4b2c-a5c0-80584960aea6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:50.275655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:50.276232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:50.276512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:50.276785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] PATH_INFO: `/v3/projects/67131ac142914dd4a311c2a30ef4228d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:50.277922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67131ac142914dd4a311c2a30ef4228d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:50.279850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:50.336119 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67131ac142914dd4a311c2a30ef4228d', 'request_id': 'req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:50.342418 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67131ac142914dd4a311c2a30ef4228d', 'request_id': 'req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:50.355435 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67131ac142914dd4a311c2a30ef4228d', 'request_id': 'req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:50.371988 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] The token cache is being invalidate because project 67131ac142914dd4a311c2a30ef4228d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:50.372375 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:50.373213 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:50.377492 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1333/2655] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:50 2022] DELETE /identity/v3/projects/67131ac142914dd4a311c2a30ef4228d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:34:50.378669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078b13f-3c4e-48e2-a823-3e9381dfb994 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:50.388489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078b13f-3c4e-48e2-a823-3e9381dfb994 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:50.423822 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:50.475601 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eac154-bc1a-445c-8740-3b83a2b155bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:50.476418 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eac154-bc1a-445c-8740-3b83a2b155bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:50.476824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eac154-bc1a-445c-8740-3b83a2b155bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:50.477171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eac154-bc1a-445c-8740-3b83a2b155bf admin admin] PATH_INFO: `/v3/users/b954bb76ed5041e6801a10f3a9dcb060` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:50.478616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eac154-bc1a-445c-8740-3b83a2b155bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b954bb76ed5041e6801a10f3a9dcb060)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:50.480753 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eac154-bc1a-445c-8740-3b83a2b155bf admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:50.553100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e1eac154-bc1a-445c-8740-3b83a2b155bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b954bb76ed5041e6801a10f3a9dcb060'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:50.570053 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e1eac154-bc1a-445c-8740-3b83a2b155bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b954bb76ed5041e6801a10f3a9dcb060'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:50.579803 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e1eac154-bc1a-445c-8740-3b83a2b155bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b954bb76ed5041e6801a10f3a9dcb060'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:50.580490 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e1eac154-bc1a-445c-8740-3b83a2b155bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b954bb76ed5041e6801a10f3a9dcb060'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:50.594405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-e1eac154-bc1a-445c-8740-3b83a2b155bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b954bb76ed5041e6801a10f3a9dcb060'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:50.602522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1323/2656] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:50 2022] DELETE /identity/v3/users/b954bb76ed5041e6801a10f3a9dcb060 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:50.994857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:50.996323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:51.013785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f0aca-e83e-40d3-97a6-ae4873a2e2b1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:51.063826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:51.090699 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d98df-bdb8-4339-8540-8c377e048c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:51.091348 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d98df-bdb8-4339-8540-8c377e048c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:51.091653 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d98df-bdb8-4339-8540-8c377e048c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:51.091978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d98df-bdb8-4339-8540-8c377e048c69 admin admin] PATH_INFO: `/v3/projects/28497684f6cb4f2fb019d1993a85b1b3` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:51.093113 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d98df-bdb8-4339-8540-8c377e048c69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28497684f6cb4f2fb019d1993a85b1b3)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:51.095066 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d98df-bdb8-4339-8540-8c377e048c69 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:51.183006 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc9d98df-bdb8-4339-8540-8c377e048c69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28497684f6cb4f2fb019d1993a85b1b3', 'request_id': 'req-fc9d98df-bdb8-4339-8540-8c377e048c69'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:51.190232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc9d98df-bdb8-4339-8540-8c377e048c69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28497684f6cb4f2fb019d1993a85b1b3', 'request_id': 'req-fc9d98df-bdb8-4339-8540-8c377e048c69'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:51.202650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc9d98df-bdb8-4339-8540-8c377e048c69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28497684f6cb4f2fb019d1993a85b1b3', 'request_id': 'req-fc9d98df-bdb8-4339-8540-8c377e048c69'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:51.217744 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc9d98df-bdb8-4339-8540-8c377e048c69 admin admin] The token cache is being invalidate because project 28497684f6cb4f2fb019d1993a85b1b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:51.218022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc9d98df-bdb8-4339-8540-8c377e048c69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:51.218717 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc9d98df-bdb8-4339-8540-8c377e048c69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:51.221357 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1334/2657] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:50 2022] DELETE /identity/v3/projects/28497684f6cb4f2fb019d1993a85b1b3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:51.226272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356aa4e-cd16-46dc-8d2c-054d8f41a207 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:51.226825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356aa4e-cd16-46dc-8d2c-054d8f41a207 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:51.226825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356aa4e-cd16-46dc-8d2c-054d8f41a207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:51.248780 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-9356aa4e-cd16-46dc-8d2c-054d8f41a207 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:51.249998 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9356aa4e-cd16-46dc-8d2c-054d8f41a207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:51.347766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1324/2658] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:51.350074 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c0cae-de8f-440c-8ceb-9c6acc170c35 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:51.350355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c0cae-de8f-440c-8ceb-9c6acc170c35 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:51.350634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c0cae-de8f-440c-8ceb-9c6acc170c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:51.371004 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-935c0cae-de8f-440c-8ceb-9c6acc170c35 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:51.372252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-935c0cae-de8f-440c-8ceb-9c6acc170c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:51.384458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1335/2659] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:51.388891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356aa4e-cd16-46dc-8d2c-054d8f41a207 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:51.401505 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356aa4e-cd16-46dc-8d2c-054d8f41a207 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:51.406613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5b2e1-e605-4e1d-8606-6106360bde33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:51.407379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5b2e1-e605-4e1d-8606-6106360bde33 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:51.407701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5b2e1-e605-4e1d-8606-6106360bde33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:51.407957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5b2e1-e605-4e1d-8606-6106360bde33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:51.411762 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5b2e1-e605-4e1d-8606-6106360bde33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:34:51.412073 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5b2e1-e605-4e1d-8606-6106360bde33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:51.413886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5b2e1-e605-4e1d-8606-6106360bde33 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:51.433636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1325/2660] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:34:51.435047 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c0cae-de8f-440c-8ceb-9c6acc170c35 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:51.445239 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c0cae-de8f-440c-8ceb-9c6acc170c35 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:51.450346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73114f73-f02d-4fde-887b-94a5c4114d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:51.451047 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73114f73-f02d-4fde-887b-94a5c4114d26 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:51.451423 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73114f73-f02d-4fde-887b-94a5c4114d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:51.451863 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73114f73-f02d-4fde-887b-94a5c4114d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:51.452584 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73114f73-f02d-4fde-887b-94a5c4114d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:51.456741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73114f73-f02d-4fde-887b-94a5c4114d26 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:51.494601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1336/2661] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:51 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:34:51.496339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5b2e1-e605-4e1d-8606-6106360bde33 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:51.509572 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5b2e1-e605-4e1d-8606-6106360bde33 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:51.555877 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:51.581324 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04939f-be0d-422d-bbce-9c3c2a4d5fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:51.582110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04939f-be0d-422d-bbce-9c3c2a4d5fbc admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:51.583081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04939f-be0d-422d-bbce-9c3c2a4d5fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:51.583451 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04939f-be0d-422d-bbce-9c3c2a4d5fbc admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:51.584246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e04939f-be0d-422d-bbce-9c3c2a4d5fbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:51.586351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e04939f-be0d-422d-bbce-9c3c2a4d5fbc admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:51.618840 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1326/2662] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:51 2022] POST /identity/v3/users => generated 284 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:51.625260 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73114f73-f02d-4fde-887b-94a5c4114d26 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:51.636850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73114f73-f02d-4fde-887b-94a5c4114d26 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:51.643220 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c90e64-6db8-4708-8c99-03726bf615f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:51.643958 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c90e64-6db8-4708-8c99-03726bf615f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:51.644296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c90e64-6db8-4708-8c99-03726bf615f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:51.644632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c90e64-6db8-4708-8c99-03726bf615f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:51.646021 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c90e64-6db8-4708-8c99-03726bf615f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:51.647699 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c90e64-6db8-4708-8c99-03726bf615f1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:51.663364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1337/2663] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:51.667158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04939f-be0d-422d-bbce-9c3c2a4d5fbc admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:51.681975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04939f-be0d-422d-bbce-9c3c2a4d5fbc admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:51.689302 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da02f94-4c93-4b52-944e-ae4bfa00e965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:51.690064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da02f94-4c93-4b52-944e-ae4bfa00e965 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:51.690617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da02f94-4c93-4b52-944e-ae4bfa00e965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:51.691122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da02f94-4c93-4b52-944e-ae4bfa00e965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:51.692777 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da02f94-4c93-4b52-944e-ae4bfa00e965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:51.694843 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da02f94-4c93-4b52-944e-ae4bfa00e965 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:51.716346 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1327/2664] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:51.720060 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c90e64-6db8-4708-8c99-03726bf615f1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:51.733712 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c90e64-6db8-4708-8c99-03726bf615f1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:51.739288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc930a-b2d7-4a55-9c77-59179ec62c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:51.740095 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc930a-b2d7-4a55-9c77-59179ec62c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:51.740472 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc930a-b2d7-4a55-9c77-59179ec62c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:51.740975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc930a-b2d7-4a55-9c77-59179ec62c58 admin admin] PATH_INFO: `/v3/projects/ef0bd62c8d9244978fa9311b449c0880/users/6c88075099924491bb4c3f076bb9efe3/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:51.759112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc930a-b2d7-4a55-9c77-59179ec62c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef0bd62c8d9244978fa9311b449c0880, user_id=6c88075099924491bb4c3f076bb9efe3, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:51.761953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc930a-b2d7-4a55-9c77-59179ec62c58 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:51.779604 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1338/2665] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:51 2022] PUT /identity/v3/projects/ef0bd62c8d9244978fa9311b449c0880/users/6c88075099924491bb4c3f076bb9efe3/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:34:51.790043 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b376047-337e-4c4c-b7e5-d1739e45f40a None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:51.790827 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b376047-337e-4c4c-b7e5-d1739e45f40a None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:51.791343 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b376047-337e-4c4c-b7e5-d1739e45f40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:51.834045 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-7b376047-337e-4c4c-b7e5-d1739e45f40a None None] MFA Rules not processed for user `6c88075099924491bb4c3f076bb9efe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:51.835438 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7b376047-337e-4c4c-b7e5-d1739e45f40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:51.925175 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1328/2666] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:51.927054 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc930a-b2d7-4a55-9c77-59179ec62c58 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:52.067023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc930a-b2d7-4a55-9c77-59179ec62c58 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:52.072905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c62b36-0dd4-42ce-a602-6c291cfa44ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:52.073891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c62b36-0dd4-42ce-a602-6c291cfa44ef admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:52.085847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c62b36-0dd4-42ce-a602-6c291cfa44ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:52.086328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c62b36-0dd4-42ce-a602-6c291cfa44ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:52.087207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c62b36-0dd4-42ce-a602-6c291cfa44ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:52.091521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c62b36-0dd4-42ce-a602-6c291cfa44ef admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:52.133401 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:52.185066 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1339/2667] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:34:51 2022] POST /identity/v3/projects => generated 365 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:34:52.188925 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b376047-337e-4c4c-b7e5-d1739e45f40a None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:52.203157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b376047-337e-4c4c-b7e5-d1739e45f40a None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:52.297691 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a77038-aea3-4744-8d88-b9236495c117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:52.298772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a77038-aea3-4744-8d88-b9236495c117 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:52.299160 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a77038-aea3-4744-8d88-b9236495c117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:52.299536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a77038-aea3-4744-8d88-b9236495c117 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:52.300334 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a77038-aea3-4744-8d88-b9236495c117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:52.302433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a77038-aea3-4744-8d88-b9236495c117 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:52.354676 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1329/2668] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:34:52 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:34:52.356087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c62b36-0dd4-42ce-a602-6c291cfa44ef admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:52.372101 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c62b36-0dd4-42ce-a602-6c291cfa44ef admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:52.386128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc33457-ab29-4f34-a06f-184ae1cb1f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:52.387046 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc33457-ab29-4f34-a06f-184ae1cb1f3a admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:52.387491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc33457-ab29-4f34-a06f-184ae1cb1f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:52.387952 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc33457-ab29-4f34-a06f-184ae1cb1f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:52.388764 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc33457-ab29-4f34-a06f-184ae1cb1f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:52.390562 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc33457-ab29-4f34-a06f-184ae1cb1f3a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:52.409604 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a77038-aea3-4744-8d88-b9236495c117 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:52.410284 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1340/2669] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:52.421866 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a77038-aea3-4744-8d88-b9236495c117 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:52.427974 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106590e4-2eb8-4b23-ba02-f1b6bb4291b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:52.429010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106590e4-2eb8-4b23-ba02-f1b6bb4291b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:52.429517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106590e4-2eb8-4b23-ba02-f1b6bb4291b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:52.429996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106590e4-2eb8-4b23-ba02-f1b6bb4291b2 admin admin] PATH_INFO: `/v3/projects/6ee0a0ac691342f0b503dc81309e9ce8/users/f9035a65e2164c3a9798d87edd3c067a/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:52.460975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106590e4-2eb8-4b23-ba02-f1b6bb4291b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee0a0ac691342f0b503dc81309e9ce8, user_id=f9035a65e2164c3a9798d87edd3c067a, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:52.464102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106590e4-2eb8-4b23-ba02-f1b6bb4291b2 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:52.492406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc33457-ab29-4f34-a06f-184ae1cb1f3a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:52.493568 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1330/2670] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:52 2022] PUT /identity/v3/projects/6ee0a0ac691342f0b503dc81309e9ce8/users/f9035a65e2164c3a9798d87edd3c067a/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:52.507838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc33457-ab29-4f34-a06f-184ae1cb1f3a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:52.587965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863085b-53e7-40c5-b1c1-831b7b044e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:52.588587 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863085b-53e7-40c5-b1c1-831b7b044e03 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:52.588871 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863085b-53e7-40c5-b1c1-831b7b044e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:52.589150 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863085b-53e7-40c5-b1c1-831b7b044e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:52.589796 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863085b-53e7-40c5-b1c1-831b7b044e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:52.591513 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863085b-53e7-40c5-b1c1-831b7b044e03 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:52.609700 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1341/2671] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:34:52.611717 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106590e4-2eb8-4b23-ba02-f1b6bb4291b2 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:52.628212 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106590e4-2eb8-4b23-ba02-f1b6bb4291b2 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:52.634289 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49579ecd-9123-4f30-bcd0-012a31a7e1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:52.635678 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49579ecd-9123-4f30-bcd0-012a31a7e1b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:52.636264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49579ecd-9123-4f30-bcd0-012a31a7e1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:52.636753 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49579ecd-9123-4f30-bcd0-012a31a7e1b6 admin admin] PATH_INFO: `/v3/projects/6ee0a0ac691342f0b503dc81309e9ce8/users/f9035a65e2164c3a9798d87edd3c067a/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:52.644524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49579ecd-9123-4f30-bcd0-012a31a7e1b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee0a0ac691342f0b503dc81309e9ce8, user_id=f9035a65e2164c3a9798d87edd3c067a, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:52.647813 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49579ecd-9123-4f30-bcd0-012a31a7e1b6 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:52.655118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:52.661740 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1331/2672] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:34:52 2022] PUT /identity/v3/projects/6ee0a0ac691342f0b503dc81309e9ce8/users/f9035a65e2164c3a9798d87edd3c067a/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:52.672697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991e5c9-903a-47a4-8cc4-3af81259ea86 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:52.672976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991e5c9-903a-47a4-8cc4-3af81259ea86 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:52.673252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991e5c9-903a-47a4-8cc4-3af81259ea86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:52.704695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0991e5c9-903a-47a4-8cc4-3af81259ea86 None None] MFA Rules not processed for user `f9035a65e2164c3a9798d87edd3c067a`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:52.705805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0991e5c9-903a-47a4-8cc4-3af81259ea86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:52.805888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1342/2673] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:52.817345 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49579ecd-9123-4f30-bcd0-012a31a7e1b6 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:52.819971 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-49579ecd-9123-4f30-bcd0-012a31a7e1b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:52.908870 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49579ecd-9123-4f30-bcd0-012a31a7e1b6 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:52.928358 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d1980-4170-4fc6-83b1-18ab22ed4ec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:52.929226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d1980-4170-4fc6-83b1-18ab22ed4ec0 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:52.930084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d1980-4170-4fc6-83b1-18ab22ed4ec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:52.930521 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d1980-4170-4fc6-83b1-18ab22ed4ec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:52.941154 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d1980-4170-4fc6-83b1-18ab22ed4ec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:52.943202 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d1980-4170-4fc6-83b1-18ab22ed4ec0 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:52.954484 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1332/2674] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:53.054155 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991e5c9-903a-47a4-8cc4-3af81259ea86 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:53.071364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991e5c9-903a-47a4-8cc4-3af81259ea86 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:53.075299 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb9011-0381-4fc0-ad5e-3d608fdd936a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:53.075955 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb9011-0381-4fc0-ad5e-3d608fdd936a service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:53.076239 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb9011-0381-4fc0-ad5e-3d608fdd936a service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:53.076514 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb9011-0381-4fc0-ad5e-3d608fdd936a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:53.164326 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb9011-0381-4fc0-ad5e-3d608fdd936a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:53.165784 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb9011-0381-4fc0-ad5e-3d608fdd936a service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:53.170631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:53.179980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1343/2675] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:55.558806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9865e62-0c24-4a65-84e7-ad55a31afee1 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:55.559328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9865e62-0c24-4a65-84e7-ad55a31afee1 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:55.559437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9865e62-0c24-4a65-84e7-ad55a31afee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:55.577632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-d9865e62-0c24-4a65-84e7-ad55a31afee1 None None] MFA Rules not processed for user `dd6241594ed040ce85c4b017cc361894`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:55.578846 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-d9865e62-0c24-4a65-84e7-ad55a31afee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:55.634586 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1333/2676] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:56.631406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb9011-0381-4fc0-ad5e-3d608fdd936a service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:56.632804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-79eb9011-0381-4fc0-ad5e-3d608fdd936a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:56.703300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb9011-0381-4fc0-ad5e-3d608fdd936a service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:56.707835 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c578098-016e-475e-a956-8919af0127cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:56.708489 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c578098-016e-475e-a956-8919af0127cb service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:56.708805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c578098-016e-475e-a956-8919af0127cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:56.709079 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c578098-016e-475e-a956-8919af0127cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:56.712056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3c578098-016e-475e-a956-8919af0127cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:56.786882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c578098-016e-475e-a956-8919af0127cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:56.787467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c578098-016e-475e-a956-8919af0127cb service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:56.798913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1344/2677] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:57.657427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c275f-a87f-4488-9b89-857106d5fe54 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:57.657924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c275f-a87f-4488-9b89-857106d5fe54 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:57.658208 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c275f-a87f-4488-9b89-857106d5fe54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:57.678015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-340c275f-a87f-4488-9b89-857106d5fe54 None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:57.679385 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-340c275f-a87f-4488-9b89-857106d5fe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:57.745701 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1334/2678] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Sep 12 23:34:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:57.759440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c578098-016e-475e-a956-8919af0127cb service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:57.760935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3c578098-016e-475e-a956-8919af0127cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:57.829817 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c578098-016e-475e-a956-8919af0127cb service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:57.835757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5f338-67fa-483d-8c5d-fd09bb12a09b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:57.836413 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5f338-67fa-483d-8c5d-fd09bb12a09b service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:57.836694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5f338-67fa-483d-8c5d-fd09bb12a09b service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:57.836971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5f338-67fa-483d-8c5d-fd09bb12a09b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:57.845313 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5f338-67fa-483d-8c5d-fd09bb12a09b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:57.846896 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5f338-67fa-483d-8c5d-fd09bb12a09b service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:57.855176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1345/2679] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:34:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:34:58.444669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c275f-a87f-4488-9b89-857106d5fe54 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:58.479072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c275f-a87f-4488-9b89-857106d5fe54 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:58.503478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b42d8-26bb-4ebe-8b0b-d2fe5cff13ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:58.504765 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b42d8-26bb-4ebe-8b0b-d2fe5cff13ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:58.504765 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b42d8-26bb-4ebe-8b0b-d2fe5cff13ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:58.505162 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b42d8-26bb-4ebe-8b0b-d2fe5cff13ce admin admin] PATH_INFO: `/v3/users/6c88075099924491bb4c3f076bb9efe3` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:58.508582 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b42d8-26bb-4ebe-8b0b-d2fe5cff13ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c88075099924491bb4c3f076bb9efe3)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:58.515473 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b42d8-26bb-4ebe-8b0b-d2fe5cff13ce admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:58.639122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f41b42d8-26bb-4ebe-8b0b-d2fe5cff13ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c88075099924491bb4c3f076bb9efe3'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:58.640337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:58.661066 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f41b42d8-26bb-4ebe-8b0b-d2fe5cff13ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c88075099924491bb4c3f076bb9efe3'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:58.671905 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f41b42d8-26bb-4ebe-8b0b-d2fe5cff13ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c88075099924491bb4c3f076bb9efe3'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:58.672904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f41b42d8-26bb-4ebe-8b0b-d2fe5cff13ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c88075099924491bb4c3f076bb9efe3'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:58.681383 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f41b42d8-26bb-4ebe-8b0b-d2fe5cff13ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c88075099924491bb4c3f076bb9efe3'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:58.695790 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1335/2680] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:58 2022] DELETE /identity/v3/users/6c88075099924491bb4c3f076bb9efe3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:58.697283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681e59e-ad6b-49e7-bee4-f132cce3174f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:58.697283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681e59e-ad6b-49e7-bee4-f132cce3174f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:58.697505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681e59e-ad6b-49e7-bee4-f132cce3174f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:58.737314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2681e59e-ad6b-49e7-bee4-f132cce3174f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:34:58.739153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2681e59e-ad6b-49e7-bee4-f132cce3174f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:58.795233 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:34:58.829652 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1346/2681] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:34:58.836914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b42d8-26bb-4ebe-8b0b-d2fe5cff13ce admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:58.838832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-f41b42d8-26bb-4ebe-8b0b-d2fe5cff13ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:58.976805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b42d8-26bb-4ebe-8b0b-d2fe5cff13ce admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:58.982023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b23327-17f8-46ac-9924-862df8a13046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:58.982732 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b23327-17f8-46ac-9924-862df8a13046 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:58.983001 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b23327-17f8-46ac-9924-862df8a13046 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:58.983272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b23327-17f8-46ac-9924-862df8a13046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:58.993221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b23327-17f8-46ac-9924-862df8a13046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:58.994744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b23327-17f8-46ac-9924-862df8a13046 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:59.019967 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1336/2682] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:34:59.553037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681e59e-ad6b-49e7-bee4-f132cce3174f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:59.554752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2681e59e-ad6b-49e7-bee4-f132cce3174f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:34:59.573061 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681e59e-ad6b-49e7-bee4-f132cce3174f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:34:59.579542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:34:59.580538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:34:59.580698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:34:59.580846 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] PATH_INFO: `/v3/projects/ef0bd62c8d9244978fa9311b449c0880` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:34:59.582565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef0bd62c8d9244978fa9311b449c0880)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:34:59.584658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:34:59.766002 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef0bd62c8d9244978fa9311b449c0880', 'request_id': 'req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:59.774338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef0bd62c8d9244978fa9311b449c0880', 'request_id': 'req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:59.785608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef0bd62c8d9244978fa9311b449c0880', 'request_id': 'req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:59.827182 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] The token cache is being invalidate because project ef0bd62c8d9244978fa9311b449c0880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:34:59.828391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:59.829942 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:34:59.833373 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1347/2683] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:34:59 2022] DELETE /identity/v3/projects/ef0bd62c8d9244978fa9311b449c0880 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:34:59.836622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b23327-17f8-46ac-9924-862df8a13046 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:34:59.849989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b23327-17f8-46ac-9924-862df8a13046 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:00.030922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6c6ce-41c5-4b63-a15a-7d99874c492f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:00.032352 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6c6ce-41c5-4b63-a15a-7d99874c492f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:00.032746 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6c6ce-41c5-4b63-a15a-7d99874c492f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:00.033124 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6c6ce-41c5-4b63-a15a-7d99874c492f admin admin] PATH_INFO: `/v3/users/f9035a65e2164c3a9798d87edd3c067a` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:00.036231 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6c6ce-41c5-4b63-a15a-7d99874c492f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9035a65e2164c3a9798d87edd3c067a)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:00.038888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6c6ce-41c5-4b63-a15a-7d99874c492f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:00.155886 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7ca6c6ce-41c5-4b63-a15a-7d99874c492f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9035a65e2164c3a9798d87edd3c067a'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:00.183779 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7ca6c6ce-41c5-4b63-a15a-7d99874c492f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9035a65e2164c3a9798d87edd3c067a'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:00.192234 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:00.208010 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7ca6c6ce-41c5-4b63-a15a-7d99874c492f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9035a65e2164c3a9798d87edd3c067a'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:00.209309 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7ca6c6ce-41c5-4b63-a15a-7d99874c492f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9035a65e2164c3a9798d87edd3c067a'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:00.219428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7ca6c6ce-41c5-4b63-a15a-7d99874c492f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9035a65e2164c3a9798d87edd3c067a'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:00.237501 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1337/2684] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:34:59 2022] DELETE /identity/v3/users/f9035a65e2164c3a9798d87edd3c067a => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:35:00.729769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:00.731225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:00.758298 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfec17-9a05-42b1-b12b-7ca6b36ccc69 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:00.855005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:00.856353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:00.856826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:00.857268 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34 admin admin] PATH_INFO: `/v3/projects/6ee0a0ac691342f0b503dc81309e9ce8` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:00.858666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ee0a0ac691342f0b503dc81309e9ce8)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:00.860847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:00.948782 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ee0a0ac691342f0b503dc81309e9ce8', 'request_id': 'req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:00.973017 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ee0a0ac691342f0b503dc81309e9ce8', 'request_id': 'req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:00.982218 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ee0a0ac691342f0b503dc81309e9ce8', 'request_id': 'req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:00.985502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:00.997806 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34 admin admin] The token cache is being invalidate because project 6ee0a0ac691342f0b503dc81309e9ce8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:35:00.998814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:00.999730 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-402f5adc-6fdf-42d2-aeb0-d13c683c6a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:01.001546 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1348/2685] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:35:00 2022] DELETE /identity/v3/projects/6ee0a0ac691342f0b503dc81309e9ce8 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:01.006555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045e7ee-60b6-4ee4-833b-ca42c8abf79f None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:01.007025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045e7ee-60b6-4ee4-833b-ca42c8abf79f None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:01.007152 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045e7ee-60b6-4ee4-833b-ca42c8abf79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:01.046935 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-5045e7ee-60b6-4ee4-833b-ca42c8abf79f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:01.048498 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5045e7ee-60b6-4ee4-833b-ca42c8abf79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:01.156062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1338/2686] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:01.165664 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda07225-705e-4b96-a545-75bdfdaf7fec None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:01.165971 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda07225-705e-4b96-a545-75bdfdaf7fec None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:01.166458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda07225-705e-4b96-a545-75bdfdaf7fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:01.212694 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dda07225-705e-4b96-a545-75bdfdaf7fec None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:01.214184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dda07225-705e-4b96-a545-75bdfdaf7fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:01.231420 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1349/2687] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:01.235519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045e7ee-60b6-4ee4-833b-ca42c8abf79f None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:01.246815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045e7ee-60b6-4ee4-833b-ca42c8abf79f None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:01.259532 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b2428-d398-4cdf-8156-a2a8157b2e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:01.260497 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b2428-d398-4cdf-8156-a2a8157b2e3f admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:01.260694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b2428-d398-4cdf-8156-a2a8157b2e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:01.260965 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b2428-d398-4cdf-8156-a2a8157b2e3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:01.261586 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b2428-d398-4cdf-8156-a2a8157b2e3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:35:01.263946 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b2428-d398-4cdf-8156-a2a8157b2e3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:01.265935 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b2428-d398-4cdf-8156-a2a8157b2e3f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:01.300084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1339/2688] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:35:01.305170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda07225-705e-4b96-a545-75bdfdaf7fec None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:01.322051 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda07225-705e-4b96-a545-75bdfdaf7fec None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:01.328007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1783b29c-7840-4294-bd89-6f5acfc1676d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:01.328855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783b29c-7840-4294-bd89-6f5acfc1676d admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:01.329369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783b29c-7840-4294-bd89-6f5acfc1676d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:01.329974 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783b29c-7840-4294-bd89-6f5acfc1676d admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:01.331000 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1783b29c-7840-4294-bd89-6f5acfc1676d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:01.334228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1783b29c-7840-4294-bd89-6f5acfc1676d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:01.406180 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1350/2689] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:35:01 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:01.407272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b2428-d398-4cdf-8156-a2a8157b2e3f admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:01.419912 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b2428-d398-4cdf-8156-a2a8157b2e3f admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:01.546293 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6556459-ef67-4bfd-b479-23ae087903e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:01.547015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556459-ef67-4bfd-b479-23ae087903e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:01.547461 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556459-ef67-4bfd-b479-23ae087903e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:01.547885 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556459-ef67-4bfd-b479-23ae087903e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:01.548599 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6556459-ef67-4bfd-b479-23ae087903e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:01.551523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6556459-ef67-4bfd-b479-23ae087903e6 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:01.596890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1783b29c-7840-4294-bd89-6f5acfc1676d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:01.603113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1340/2690] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:35:01 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:01.615958 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1783b29c-7840-4294-bd89-6f5acfc1676d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:01.621641 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd25a1-5bee-4b69-bcf8-46011f74ecf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:01.631460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd25a1-5bee-4b69-bcf8-46011f74ecf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:01.631460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd25a1-5bee-4b69-bcf8-46011f74ecf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:01.634498 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd25a1-5bee-4b69-bcf8-46011f74ecf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:01.635346 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd25a1-5bee-4b69-bcf8-46011f74ecf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:01.638899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd25a1-5bee-4b69-bcf8-46011f74ecf7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:01.659053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1351/2691] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:35:01.663424 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6556459-ef67-4bfd-b479-23ae087903e6 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:01.674829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6556459-ef67-4bfd-b479-23ae087903e6 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:01.680314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b5928-97df-4d7c-99af-6662d2602912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:01.692365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b5928-97df-4d7c-99af-6662d2602912 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:01.692744 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b5928-97df-4d7c-99af-6662d2602912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:01.693437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b5928-97df-4d7c-99af-6662d2602912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:01.694357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b5928-97df-4d7c-99af-6662d2602912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:01.697503 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b5928-97df-4d7c-99af-6662d2602912 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:01.720685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1341/2692] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:35:01.725854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd25a1-5bee-4b69-bcf8-46011f74ecf7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:01.748722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd25a1-5bee-4b69-bcf8-46011f74ecf7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:01.765444 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22803fc3-fc16-4484-8fe2-65c9e8d6318b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:01.766134 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22803fc3-fc16-4484-8fe2-65c9e8d6318b admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:01.766416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22803fc3-fc16-4484-8fe2-65c9e8d6318b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:01.766961 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22803fc3-fc16-4484-8fe2-65c9e8d6318b admin admin] PATH_INFO: `/v3/projects/d8a0d97d4c2b413b9318df6d71d0ad62/users/53e736714ee845d699a7d41627212e57/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:01.795590 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22803fc3-fc16-4484-8fe2-65c9e8d6318b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a0d97d4c2b413b9318df6d71d0ad62, user_id=53e736714ee845d699a7d41627212e57, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:01.799063 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22803fc3-fc16-4484-8fe2-65c9e8d6318b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:01.819080 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1352/2693] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:35:01 2022] PUT /identity/v3/projects/d8a0d97d4c2b413b9318df6d71d0ad62/users/53e736714ee845d699a7d41627212e57/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:01.829655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f8fa1-877b-4186-9503-9a218989ef58 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:01.829945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f8fa1-877b-4186-9503-9a218989ef58 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:01.830228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f8fa1-877b-4186-9503-9a218989ef58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:01.908707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-eb9f8fa1-877b-4186-9503-9a218989ef58 None None] MFA Rules not processed for user `53e736714ee845d699a7d41627212e57`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:01.910524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-eb9f8fa1-877b-4186-9503-9a218989ef58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:02.024730 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:02.103183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1342/2694] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:02.104443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22803fc3-fc16-4484-8fe2-65c9e8d6318b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:02.208691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22803fc3-fc16-4484-8fe2-65c9e8d6318b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:02.221685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b39940-d31e-4d1a-bfbe-f1dde1ee88b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:02.223455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b39940-d31e-4d1a-bfbe-f1dde1ee88b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:02.223781 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b39940-d31e-4d1a-bfbe-f1dde1ee88b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:02.224140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b39940-d31e-4d1a-bfbe-f1dde1ee88b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:02.226773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b39940-d31e-4d1a-bfbe-f1dde1ee88b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:02.227713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b39940-d31e-4d1a-bfbe-f1dde1ee88b9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:02.275169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1353/2695] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:35:02 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:02.279004 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f8fa1-877b-4186-9503-9a218989ef58 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:02.296502 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f8fa1-877b-4186-9503-9a218989ef58 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:02.309810 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b39940-d31e-4d1a-bfbe-f1dde1ee88b9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:02.312300 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d2b39940-d31e-4d1a-bfbe-f1dde1ee88b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:02.385357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ffad34-175e-4bad-99ca-a0a3a9b62c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:02.387256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffad34-175e-4bad-99ca-a0a3a9b62c15 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:02.387256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffad34-175e-4bad-99ca-a0a3a9b62c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:02.387672 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffad34-175e-4bad-99ca-a0a3a9b62c15 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:02.388476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ffad34-175e-4bad-99ca-a0a3a9b62c15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:02.391523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ffad34-175e-4bad-99ca-a0a3a9b62c15 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:02.422555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b39940-d31e-4d1a-bfbe-f1dde1ee88b9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:02.430503 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f7e32-bc28-4ee4-9579-31edbf6c1daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:02.431217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f7e32-bc28-4ee4-9579-31edbf6c1daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:02.431766 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f7e32-bc28-4ee4-9579-31edbf6c1daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:02.432118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f7e32-bc28-4ee4-9579-31edbf6c1daa admin admin] PATH_INFO: `/v3/users/2749a61b3dd04d89a5c2e4235aa96934` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:02.435246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f7e32-bc28-4ee4-9579-31edbf6c1daa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2749a61b3dd04d89a5c2e4235aa96934)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:02.437724 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f7e32-bc28-4ee4-9579-31edbf6c1daa admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:02.441948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1343/2696] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:35:02 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:02.445966 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ffad34-175e-4bad-99ca-a0a3a9b62c15 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:02.457744 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:02.470467 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ffad34-175e-4bad-99ca-a0a3a9b62c15 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:02.491274 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a47c93-6891-458d-a449-d03923968032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:02.491935 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a47c93-6891-458d-a449-d03923968032 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:02.492219 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a47c93-6891-458d-a449-d03923968032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:02.492500 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a47c93-6891-458d-a449-d03923968032 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:02.493150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a47c93-6891-458d-a449-d03923968032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:02.494781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a47c93-6891-458d-a449-d03923968032 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:02.509084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1344/2697] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:35:02.514585 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a47c93-6891-458d-a449-d03923968032 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:02.515898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad2f7e32-bc28-4ee4-9579-31edbf6c1daa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2749a61b3dd04d89a5c2e4235aa96934'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:02.529039 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad2f7e32-bc28-4ee4-9579-31edbf6c1daa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2749a61b3dd04d89a5c2e4235aa96934'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:02.545622 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad2f7e32-bc28-4ee4-9579-31edbf6c1daa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2749a61b3dd04d89a5c2e4235aa96934'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:02.547740 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a47c93-6891-458d-a449-d03923968032 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:02.556527 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad2f7e32-bc28-4ee4-9579-31edbf6c1daa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2749a61b3dd04d89a5c2e4235aa96934'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:02.558140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ad2f7e32-bc28-4ee4-9579-31edbf6c1daa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2749a61b3dd04d89a5c2e4235aa96934'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:02.561457 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d3c33-4198-443b-8aff-dbb641d88d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:02.564171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d3c33-4198-443b-8aff-dbb641d88d7c admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:02.565812 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d3c33-4198-443b-8aff-dbb641d88d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:02.566624 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d3c33-4198-443b-8aff-dbb641d88d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:02.567906 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d3c33-4198-443b-8aff-dbb641d88d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:02.571407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d3c33-4198-443b-8aff-dbb641d88d7c admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:02.577575 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1354/2698] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:35:02 2022] DELETE /identity/v3/users/2749a61b3dd04d89a5c2e4235aa96934 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:02.588809 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1345/2699] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:35:02.593505 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f7e32-bc28-4ee4-9579-31edbf6c1daa admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:02.595053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ad2f7e32-bc28-4ee4-9579-31edbf6c1daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:02.702001 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f7e32-bc28-4ee4-9579-31edbf6c1daa admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:02.714355 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81dd787-97c8-44ac-b595-5837e68ed27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:02.715022 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81dd787-97c8-44ac-b595-5837e68ed27a admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:02.715304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81dd787-97c8-44ac-b595-5837e68ed27a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:02.715773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81dd787-97c8-44ac-b595-5837e68ed27a admin admin] PATH_INFO: `/v3/projects/65481cfb1e414a48a1b55be61b18629b/users/0c495a289dda46658d4195d42af56a73/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:02.733854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81dd787-97c8-44ac-b595-5837e68ed27a admin admin] RBAC: Authorizing `identity:create_grant(project_id=65481cfb1e414a48a1b55be61b18629b, user_id=0c495a289dda46658d4195d42af56a73, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:02.737706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81dd787-97c8-44ac-b595-5837e68ed27a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:02.760579 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1355/2700] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:35:02 2022] PUT /identity/v3/projects/65481cfb1e414a48a1b55be61b18629b/users/0c495a289dda46658d4195d42af56a73/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:02.835984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feddf23-c61a-435e-94f4-57f4f15f5c28 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:02.836281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feddf23-c61a-435e-94f4-57f4f15f5c28 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:02.836584 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feddf23-c61a-435e-94f4-57f4f15f5c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:02.879897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-5feddf23-c61a-435e-94f4-57f4f15f5c28 None None] MFA Rules not processed for user `0c495a289dda46658d4195d42af56a73`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:02.881251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-5feddf23-c61a-435e-94f4-57f4f15f5c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:02.964667 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81dd787-97c8-44ac-b595-5837e68ed27a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:02.965114 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1346/2701] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:02.966086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feddf23-c61a-435e-94f4-57f4f15f5c28 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:02.967293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f81dd787-97c8-44ac-b595-5837e68ed27a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:03.025836 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:03.067693 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feddf23-c61a-435e-94f4-57f4f15f5c28 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:03.074604 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81dd787-97c8-44ac-b595-5837e68ed27a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:03.076837 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b6896-7936-4bc2-8ea6-cfcdd9a2de9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:03.077486 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b6896-7936-4bc2-8ea6-cfcdd9a2de9a admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.077760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b6896-7936-4bc2-8ea6-cfcdd9a2de9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.078042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b6896-7936-4bc2-8ea6-cfcdd9a2de9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.079064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b6896-7936-4bc2-8ea6-cfcdd9a2de9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:03.079601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d5945-bc83-44cc-ba37-b8695a94f7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:03.080318 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d5945-bc83-44cc-ba37-b8695a94f7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.080630 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d5945-bc83-44cc-ba37-b8695a94f7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.080868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d5945-bc83-44cc-ba37-b8695a94f7ff admin admin] PATH_INFO: `/v3/projects/5aa1c574bdd14a1bb0ae5f82f892e6bf` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.081082 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b6896-7936-4bc2-8ea6-cfcdd9a2de9a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:03.082026 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d5945-bc83-44cc-ba37-b8695a94f7ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aa1c574bdd14a1bb0ae5f82f892e6bf)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:03.084266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d5945-bc83-44cc-ba37-b8695a94f7ff admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:03.111283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:03.126207 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1347/2702] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:35:02 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:03.128100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b6896-7936-4bc2-8ea6-cfcdd9a2de9a admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:03.156410 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e20d5945-bc83-44cc-ba37-b8695a94f7ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aa1c574bdd14a1bb0ae5f82f892e6bf', 'request_id': 'req-e20d5945-bc83-44cc-ba37-b8695a94f7ff'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:03.170613 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e20d5945-bc83-44cc-ba37-b8695a94f7ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aa1c574bdd14a1bb0ae5f82f892e6bf', 'request_id': 'req-e20d5945-bc83-44cc-ba37-b8695a94f7ff'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:03.180059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e20d5945-bc83-44cc-ba37-b8695a94f7ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aa1c574bdd14a1bb0ae5f82f892e6bf', 'request_id': 'req-e20d5945-bc83-44cc-ba37-b8695a94f7ff'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:03.196932 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e20d5945-bc83-44cc-ba37-b8695a94f7ff admin admin] The token cache is being invalidate because project 5aa1c574bdd14a1bb0ae5f82f892e6bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:35:03.197251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e20d5945-bc83-44cc-ba37-b8695a94f7ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:03.197938 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e20d5945-bc83-44cc-ba37-b8695a94f7ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:03.198411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b6896-7936-4bc2-8ea6-cfcdd9a2de9a admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:03.202894 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1356/2703] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:35:02 2022] DELETE /identity/v3/projects/5aa1c574bdd14a1bb0ae5f82f892e6bf => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:03.205630 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e43fdd-e232-42aa-954b-a395395bae12 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.205918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e43fdd-e232-42aa-954b-a395395bae12 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.206197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e43fdd-e232-42aa-954b-a395395bae12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.233339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-88e43fdd-e232-42aa-954b-a395395bae12 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:03.234591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-88e43fdd-e232-42aa-954b-a395395bae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:03.281845 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75569c-db23-40b3-ba93-b94a20d8d372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:03.282750 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75569c-db23-40b3-ba93-b94a20d8d372 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.283037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75569c-db23-40b3-ba93-b94a20d8d372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.283312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75569c-db23-40b3-ba93-b94a20d8d372 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.283989 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75569c-db23-40b3-ba93-b94a20d8d372 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:03.286035 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75569c-db23-40b3-ba93-b94a20d8d372 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:03.314521 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1348/2704] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:35:03 2022] POST /identity/v3/users => generated 299 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:03.320282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75569c-db23-40b3-ba93-b94a20d8d372 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:03.326481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1e75569c-db23-40b3-ba93-b94a20d8d372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:03.347811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1357/2705] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:35:03.351336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca552e-61ff-4ad3-b704-0247aa8d1770 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.351648 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca552e-61ff-4ad3-b704-0247aa8d1770 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.351926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca552e-61ff-4ad3-b704-0247aa8d1770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.352496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75569c-db23-40b3-ba93-b94a20d8d372 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:03.357930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b365b0-a672-475e-aebe-2e12a9dbb58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:03.358569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b365b0-a672-475e-aebe-2e12a9dbb58b admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.358830 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b365b0-a672-475e-aebe-2e12a9dbb58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.359214 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b365b0-a672-475e-aebe-2e12a9dbb58b admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.359899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b365b0-a672-475e-aebe-2e12a9dbb58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:03.362012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b365b0-a672-475e-aebe-2e12a9dbb58b admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:03.376397 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1349/2706] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:35:03.380353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a2ca552e-61ff-4ad3-b704-0247aa8d1770 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:03.381572 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a2ca552e-61ff-4ad3-b704-0247aa8d1770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:03.381703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b365b0-a672-475e-aebe-2e12a9dbb58b admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:03.390304 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1358/2707] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:35:03.394332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca552e-61ff-4ad3-b704-0247aa8d1770 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:03.395862 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b365b0-a672-475e-aebe-2e12a9dbb58b admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:03.402617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104616e0-124e-4a01-b1f6-f9690b511118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:03.403186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104616e0-124e-4a01-b1f6-f9690b511118 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.403463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104616e0-124e-4a01-b1f6-f9690b511118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.403781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104616e0-124e-4a01-b1f6-f9690b511118 admin admin] PATH_INFO: `/v3/projects/e821263aefb749d49f94c08266fc8abf/users/8aaeec6b60434c4ba8280111b0eb15d3/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.405639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca552e-61ff-4ad3-b704-0247aa8d1770 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:03.410558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c296c-4b7a-4419-9371-2fc606b42d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:03.411161 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c296c-4b7a-4419-9371-2fc606b42d3d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.411442 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c296c-4b7a-4419-9371-2fc606b42d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.411752 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c296c-4b7a-4419-9371-2fc606b42d3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.412563 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c296c-4b7a-4419-9371-2fc606b42d3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:35:03.412870 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c296c-4b7a-4419-9371-2fc606b42d3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:03.414471 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c296c-4b7a-4419-9371-2fc606b42d3d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:03.419891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104616e0-124e-4a01-b1f6-f9690b511118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e821263aefb749d49f94c08266fc8abf, user_id=8aaeec6b60434c4ba8280111b0eb15d3, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:03.422540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104616e0-124e-4a01-b1f6-f9690b511118 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:03.433459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1359/2708] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:35:03.438262 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c296c-4b7a-4419-9371-2fc606b42d3d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:03.442820 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1350/2709] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:35:03 2022] PUT /identity/v3/projects/e821263aefb749d49f94c08266fc8abf/users/8aaeec6b60434c4ba8280111b0eb15d3/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:03.443887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104616e0-124e-4a01-b1f6-f9690b511118 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:03.450903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c296c-4b7a-4419-9371-2fc606b42d3d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:03.482132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:03.524953 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104616e0-124e-4a01-b1f6-f9690b511118 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:03.528499 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6413db-5aa8-488b-9b4a-767df18f8523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:03.529330 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6413db-5aa8-488b-9b4a-767df18f8523 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.529755 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6413db-5aa8-488b-9b4a-767df18f8523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.530037 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6413db-5aa8-488b-9b4a-767df18f8523 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.530592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4314d-d582-4c5e-a40b-55ea0d85ee92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:03.530975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6413db-5aa8-488b-9b4a-767df18f8523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:03.531696 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4314d-d582-4c5e-a40b-55ea0d85ee92 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.532209 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4314d-d582-4c5e-a40b-55ea0d85ee92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.532725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4314d-d582-4c5e-a40b-55ea0d85ee92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.533087 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6413db-5aa8-488b-9b4a-767df18f8523 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:03.533805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4314d-d582-4c5e-a40b-55ea0d85ee92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:03.536371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4314d-d582-4c5e-a40b-55ea0d85ee92 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:03.551171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1351/2710] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:35:03.556593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4314d-d582-4c5e-a40b-55ea0d85ee92 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:03.567388 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1360/2711] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:35:03 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:03.568880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4314d-d582-4c5e-a40b-55ea0d85ee92 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:03.573130 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6413db-5aa8-488b-9b4a-767df18f8523 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:03.596120 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:03.644671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fc1ee-50b3-4085-a849-446ab852bc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:03.645348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fc1ee-50b3-4085-a849-446ab852bc6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.645632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fc1ee-50b3-4085-a849-446ab852bc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.645913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fc1ee-50b3-4085-a849-446ab852bc6a admin admin] PATH_INFO: `/v3/projects/e821263aefb749d49f94c08266fc8abf/users/8aaeec6b60434c4ba8280111b0eb15d3/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.649025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fc1ee-50b3-4085-a849-446ab852bc6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e821263aefb749d49f94c08266fc8abf, user_id=8aaeec6b60434c4ba8280111b0eb15d3, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:03.651828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fc1ee-50b3-4085-a849-446ab852bc6a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:03.663244 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6413db-5aa8-488b-9b4a-767df18f8523 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:03.669138 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1352/2712] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:35:03 2022] PUT /identity/v3/projects/e821263aefb749d49f94c08266fc8abf/users/8aaeec6b60434c4ba8280111b0eb15d3/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:03.680177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84758c-662d-494c-be4f-dd4d76a40f74 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.680494 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84758c-662d-494c-be4f-dd4d76a40f74 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.680735 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84758c-662d-494c-be4f-dd4d76a40f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.726334 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-9f84758c-662d-494c-be4f-dd4d76a40f74 None None] MFA Rules not processed for user `8aaeec6b60434c4ba8280111b0eb15d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:03.727602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9f84758c-662d-494c-be4f-dd4d76a40f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:03.759309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cecca-b964-4f6a-a19b-2f733de1974d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:03.759950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cecca-b964-4f6a-a19b-2f733de1974d admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.760228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cecca-b964-4f6a-a19b-2f733de1974d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.760559 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cecca-b964-4f6a-a19b-2f733de1974d admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.761332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cecca-b964-4f6a-a19b-2f733de1974d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:03.764101 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cecca-b964-4f6a-a19b-2f733de1974d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:03.797857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1361/2713] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:35:03 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:03.802969 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cecca-b964-4f6a-a19b-2f733de1974d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:03.816102 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cecca-b964-4f6a-a19b-2f733de1974d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:03.820836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc411998-744a-4136-8aab-81e7b1541b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:03.821443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc411998-744a-4136-8aab-81e7b1541b64 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.821728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc411998-744a-4136-8aab-81e7b1541b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.822047 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc411998-744a-4136-8aab-81e7b1541b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.822710 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc411998-744a-4136-8aab-81e7b1541b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:03.824353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc411998-744a-4136-8aab-81e7b1541b64 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:03.833550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1353/2714] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:35:03.834668 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1362/2715] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:35:03.839202 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84758c-662d-494c-be4f-dd4d76a40f74 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:03.850295 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc411998-744a-4136-8aab-81e7b1541b64 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:03.850587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84758c-662d-494c-be4f-dd4d76a40f74 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:03.852558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bc411998-744a-4136-8aab-81e7b1541b64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:03.855696 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092809a-eb55-4491-8822-8a8b1df3e009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:03.856426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092809a-eb55-4491-8822-8a8b1df3e009 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.856651 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092809a-eb55-4491-8822-8a8b1df3e009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.856922 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092809a-eb55-4491-8822-8a8b1df3e009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.857573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092809a-eb55-4491-8822-8a8b1df3e009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:03.859333 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092809a-eb55-4491-8822-8a8b1df3e009 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:03.878888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1354/2716] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:35:03.879829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092809a-eb55-4491-8822-8a8b1df3e009 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:03.891669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092809a-eb55-4491-8822-8a8b1df3e009 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:03.903273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183abb2-6a67-4e6b-8922-145832b718d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:03.904222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183abb2-6a67-4e6b-8922-145832b718d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.905539 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183abb2-6a67-4e6b-8922-145832b718d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.907775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183abb2-6a67-4e6b-8922-145832b718d4 admin admin] PATH_INFO: `/v3/projects/1649d6cd95d4496baaee3a0c0595a304/users/81501360f082464191731f57d2b6a402/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.947617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183abb2-6a67-4e6b-8922-145832b718d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1649d6cd95d4496baaee3a0c0595a304, user_id=81501360f082464191731f57d2b6a402, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:03.947851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc411998-744a-4136-8aab-81e7b1541b64 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:03.948199 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183abb2-6a67-4e6b-8922-145832b718d4 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:03.952503 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf2dab-402b-40c6-897f-7ad14b5a7a6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:03.953616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf2dab-402b-40c6-897f-7ad14b5a7a6d service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.953616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf2dab-402b-40c6-897f-7ad14b5a7a6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.953851 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf2dab-402b-40c6-897f-7ad14b5a7a6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:03.960875 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-84cf2dab-402b-40c6-897f-7ad14b5a7a6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:03.979313 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1355/2717] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:35:03 2022] PUT /identity/v3/projects/1649d6cd95d4496baaee3a0c0595a304/users/81501360f082464191731f57d2b6a402/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:03.985738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c613cbb-579b-40d6-b64f-e1c7f1d322be None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:03.986065 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c613cbb-579b-40d6-b64f-e1c7f1d322be None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:03.992683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c613cbb-579b-40d6-b64f-e1c7f1d322be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:04.015464 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:04.029113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-6c613cbb-579b-40d6-b64f-e1c7f1d322be None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:04.033501 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6c613cbb-579b-40d6-b64f-e1c7f1d322be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:04.080772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:04.088101 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf2dab-402b-40c6-897f-7ad14b5a7a6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:04.090276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf2dab-402b-40c6-897f-7ad14b5a7a6d service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:04.105033 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1363/2718] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:35:04.157946 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1356/2719] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:04.158807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf2dab-402b-40c6-897f-7ad14b5a7a6d service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:04.160861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-84cf2dab-402b-40c6-897f-7ad14b5a7a6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:04.244879 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf2dab-402b-40c6-897f-7ad14b5a7a6d service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:04.311324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb164b1-06f1-42fd-bdbd-d82b37987b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:04.312382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb164b1-06f1-42fd-bdbd-d82b37987b2e service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:04.313946 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb164b1-06f1-42fd-bdbd-d82b37987b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:04.313946 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb164b1-06f1-42fd-bdbd-d82b37987b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:04.327951 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb164b1-06f1-42fd-bdbd-d82b37987b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:04.329581 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb164b1-06f1-42fd-bdbd-d82b37987b2e service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:04.340994 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1364/2720] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:35:05.896988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c613cbb-579b-40d6-b64f-e1c7f1d322be None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:05.899518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-6c613cbb-579b-40d6-b64f-e1c7f1d322be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:05.992339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c613cbb-579b-40d6-b64f-e1c7f1d322be None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:06.000316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3877a2c-af9a-4bbe-8788-ac38ba02f98f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:06.000316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3877a2c-af9a-4bbe-8788-ac38ba02f98f service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:06.000316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3877a2c-af9a-4bbe-8788-ac38ba02f98f service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:06.000316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3877a2c-af9a-4bbe-8788-ac38ba02f98f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:06.004227 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e3877a2c-af9a-4bbe-8788-ac38ba02f98f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:06.109775 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3877a2c-af9a-4bbe-8788-ac38ba02f98f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:06.115601 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3877a2c-af9a-4bbe-8788-ac38ba02f98f service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:06.150168 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1357/2721] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:35:07.015112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb164b1-06f1-42fd-bdbd-d82b37987b2e service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:07.025715 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb164b1-06f1-42fd-bdbd-d82b37987b2e service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:07.029914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeab93d-97d1-47fd-a6d4-6de5b9395d97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:07.030533 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeab93d-97d1-47fd-a6d4-6de5b9395d97 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:07.030812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeab93d-97d1-47fd-a6d4-6de5b9395d97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:07.031112 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeab93d-97d1-47fd-a6d4-6de5b9395d97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:07.103433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeab93d-97d1-47fd-a6d4-6de5b9395d97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:07.105070 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeab93d-97d1-47fd-a6d4-6de5b9395d97 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:07.130216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1365/2722] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:35:09.491959 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aeb5e-c415-4373-8895-4134ac3b9dd0 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:09.492533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aeb5e-c415-4373-8895-4134ac3b9dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:09.493042 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aeb5e-c415-4373-8895-4134ac3b9dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:09.525493 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-639aeb5e-c415-4373-8895-4134ac3b9dd0 None None] MFA Rules not processed for user `81501360f082464191731f57d2b6a402`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:09.526866 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-639aeb5e-c415-4373-8895-4134ac3b9dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:09.599041 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:09.622860 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1358/2723] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:09.638228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeab93d-97d1-47fd-a6d4-6de5b9395d97 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:09.639801 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-feeab93d-97d1-47fd-a6d4-6de5b9395d97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:09.695139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:09.725053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeab93d-97d1-47fd-a6d4-6de5b9395d97 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:09.731069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e628f-4211-4432-baa9-ebdecc017f9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:09.731799 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e628f-4211-4432-baa9-ebdecc017f9a service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:09.732084 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e628f-4211-4432-baa9-ebdecc017f9a service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:09.732360 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e628f-4211-4432-baa9-ebdecc017f9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:09.742069 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e628f-4211-4432-baa9-ebdecc017f9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:09.743620 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e628f-4211-4432-baa9-ebdecc017f9a service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:09.755268 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1366/2724] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:35:11.282865 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39422c0-9f6a-456a-bc44-11e47007d0ea None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:11.283427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39422c0-9f6a-456a-bc44-11e47007d0ea None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:11.283427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39422c0-9f6a-456a-bc44-11e47007d0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:11.304456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c39422c0-9f6a-456a-bc44-11e47007d0ea None None] MFA Rules not processed for user `81501360f082464191731f57d2b6a402`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:11.305697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c39422c0-9f6a-456a-bc44-11e47007d0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:11.313276 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1359/2725] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:11.497173 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e628f-4211-4432-baa9-ebdecc017f9a service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:11.510891 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e628f-4211-4432-baa9-ebdecc017f9a service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:11.516657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e44bb-061e-4c90-b354-585d42b40e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:11.517462 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e44bb-061e-4c90-b354-585d42b40e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:11.517935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e44bb-061e-4c90-b354-585d42b40e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:11.518213 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e44bb-061e-4c90-b354-585d42b40e93 admin admin] PATH_INFO: `/v3/users/53e736714ee845d699a7d41627212e57` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:11.519947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e44bb-061e-4c90-b354-585d42b40e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e736714ee845d699a7d41627212e57)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:11.521874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e44bb-061e-4c90-b354-585d42b40e93 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:11.587096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c88e44bb-061e-4c90-b354-585d42b40e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e736714ee845d699a7d41627212e57'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:11.600606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c88e44bb-061e-4c90-b354-585d42b40e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e736714ee845d699a7d41627212e57'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:11.620804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c88e44bb-061e-4c90-b354-585d42b40e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e736714ee845d699a7d41627212e57'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:11.633477 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c88e44bb-061e-4c90-b354-585d42b40e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e736714ee845d699a7d41627212e57'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:11.635393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c88e44bb-061e-4c90-b354-585d42b40e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e736714ee845d699a7d41627212e57'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:11.646283 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1367/2726] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:35:11 2022] DELETE /identity/v3/users/53e736714ee845d699a7d41627212e57 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:11.651028 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cb5c5-2635-4fb7-9b40-c541f107d179 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:11.651314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cb5c5-2635-4fb7-9b40-c541f107d179 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:11.651639 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cb5c5-2635-4fb7-9b40-c541f107d179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:11.671608 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-1e7cb5c5-2635-4fb7-9b40-c541f107d179 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:11.672943 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1e7cb5c5-2635-4fb7-9b40-c541f107d179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:11.690229 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e44bb-061e-4c90-b354-585d42b40e93 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:11.691964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c88e44bb-061e-4c90-b354-585d42b40e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:11.739391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1360/2727] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:11.749099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cb5c5-2635-4fb7-9b40-c541f107d179 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:11.750585 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1e7cb5c5-2635-4fb7-9b40-c541f107d179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:11.763169 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e44bb-061e-4c90-b354-585d42b40e93 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:11.764602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cb5c5-2635-4fb7-9b40-c541f107d179 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:11.767542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bd97e-375d-4532-8395-e3467879465b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:11.768359 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cffda2-1a41-41fb-a0a7-8869f223080d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:11.769002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cffda2-1a41-41fb-a0a7-8869f223080d service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:11.769242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bd97e-375d-4532-8395-e3467879465b service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:11.769377 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cffda2-1a41-41fb-a0a7-8869f223080d service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:11.769564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bd97e-375d-4532-8395-e3467879465b service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:11.769671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cffda2-1a41-41fb-a0a7-8869f223080d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:11.769843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bd97e-375d-4532-8395-e3467879465b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:11.771857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-df5bd97e-375d-4532-8395-e3467879465b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:11.779290 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cffda2-1a41-41fb-a0a7-8869f223080d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:11.780786 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cffda2-1a41-41fb-a0a7-8869f223080d service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:11.784383 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bd97e-375d-4532-8395-e3467879465b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:11.786297 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bd97e-375d-4532-8395-e3467879465b service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:11.791386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1361/2728] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:35:11.800378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1368/2729] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:35:12.260533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cffda2-1a41-41fb-a0a7-8869f223080d service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:12.262150 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-54cffda2-1a41-41fb-a0a7-8869f223080d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:12.283872 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cffda2-1a41-41fb-a0a7-8869f223080d service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:12.318365 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:12.319528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:12.320049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:12.320727 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] PATH_INFO: `/v3/projects/d8a0d97d4c2b413b9318df6d71d0ad62` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:12.335848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8a0d97d4c2b413b9318df6d71d0ad62)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:12.336254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:12.411716 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8a0d97d4c2b413b9318df6d71d0ad62', 'request_id': 'req-18576338-48aa-47b2-b79c-ab49ec8b18ae'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:12.426089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8a0d97d4c2b413b9318df6d71d0ad62', 'request_id': 'req-18576338-48aa-47b2-b79c-ab49ec8b18ae'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:12.438059 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8a0d97d4c2b413b9318df6d71d0ad62', 'request_id': 'req-18576338-48aa-47b2-b79c-ab49ec8b18ae'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:12.458347 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] The token cache is being invalidate because project d8a0d97d4c2b413b9318df6d71d0ad62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:35:12.458872 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:12.459642 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:12.461534 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1362/2730] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:35:12 2022] DELETE /identity/v3/projects/d8a0d97d4c2b413b9318df6d71d0ad62 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:12.465468 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bd97e-375d-4532-8395-e3467879465b service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:12.489267 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bd97e-375d-4532-8395-e3467879465b service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:12.549565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:12.569769 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9907e-2ffb-4e89-b4ac-bc11b69c6729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:12.570350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9907e-2ffb-4e89-b4ac-bc11b69c6729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:12.570632 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9907e-2ffb-4e89-b4ac-bc11b69c6729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:12.570907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9907e-2ffb-4e89-b4ac-bc11b69c6729 admin admin] PATH_INFO: `/v3/users/0c495a289dda46658d4195d42af56a73` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:12.572084 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9907e-2ffb-4e89-b4ac-bc11b69c6729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c495a289dda46658d4195d42af56a73)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:12.574417 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9907e-2ffb-4e89-b4ac-bc11b69c6729 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:12.664569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1eb9907e-2ffb-4e89-b4ac-bc11b69c6729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c495a289dda46658d4195d42af56a73'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:12.672521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1eb9907e-2ffb-4e89-b4ac-bc11b69c6729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c495a289dda46658d4195d42af56a73'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:12.693993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1eb9907e-2ffb-4e89-b4ac-bc11b69c6729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c495a289dda46658d4195d42af56a73'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:12.711950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1eb9907e-2ffb-4e89-b4ac-bc11b69c6729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c495a289dda46658d4195d42af56a73'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:12.712680 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1eb9907e-2ffb-4e89-b4ac-bc11b69c6729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c495a289dda46658d4195d42af56a73'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:12.727945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1369/2731] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:35:12 2022] DELETE /identity/v3/users/0c495a289dda46658d4195d42af56a73 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:13.064367 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:13.066476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:13.087480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18576338-48aa-47b2-b79c-ab49ec8b18ae admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:13.177620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:13.185845 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:13.186554 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:13.187090 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:13.187540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] PATH_INFO: `/v3/projects/65481cfb1e414a48a1b55be61b18629b` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:13.189139 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] RBAC: Authorizing `identity:delete_project(project_id=65481cfb1e414a48a1b55be61b18629b)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:13.193205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:13.251018 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65481cfb1e414a48a1b55be61b18629b', 'request_id': 'req-20b78240-4aaa-446e-95e8-8a1637a798af'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:13.257710 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65481cfb1e414a48a1b55be61b18629b', 'request_id': 'req-20b78240-4aaa-446e-95e8-8a1637a798af'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:13.272940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65481cfb1e414a48a1b55be61b18629b', 'request_id': 'req-20b78240-4aaa-446e-95e8-8a1637a798af'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:13.288876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] The token cache is being invalidate because project 65481cfb1e414a48a1b55be61b18629b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:35:13.289958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:13.290855 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:13.296303 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1363/2732] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:35:13 2022] DELETE /identity/v3/projects/65481cfb1e414a48a1b55be61b18629b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:13.297212 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9907e-2ffb-4e89-b4ac-bc11b69c6729 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:13.299096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1eb9907e-2ffb-4e89-b4ac-bc11b69c6729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:13.322578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9907e-2ffb-4e89-b4ac-bc11b69c6729 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:13.423961 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84e7db-be16-471f-bfc9-6493dcba0a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:13.424892 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84e7db-be16-471f-bfc9-6493dcba0a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:13.426182 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84e7db-be16-471f-bfc9-6493dcba0a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:13.426543 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84e7db-be16-471f-bfc9-6493dcba0a22 admin admin] PATH_INFO: `/v3/users/8aaeec6b60434c4ba8280111b0eb15d3` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:13.427980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84e7db-be16-471f-bfc9-6493dcba0a22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aaeec6b60434c4ba8280111b0eb15d3)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:13.432043 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84e7db-be16-471f-bfc9-6493dcba0a22 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:13.496183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0a84e7db-be16-471f-bfc9-6493dcba0a22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aaeec6b60434c4ba8280111b0eb15d3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:13.506812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0a84e7db-be16-471f-bfc9-6493dcba0a22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aaeec6b60434c4ba8280111b0eb15d3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:13.516936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:13.532234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0a84e7db-be16-471f-bfc9-6493dcba0a22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aaeec6b60434c4ba8280111b0eb15d3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:13.548455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0a84e7db-be16-471f-bfc9-6493dcba0a22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aaeec6b60434c4ba8280111b0eb15d3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:13.550736 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0a84e7db-be16-471f-bfc9-6493dcba0a22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aaeec6b60434c4ba8280111b0eb15d3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:13.567625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1370/2733] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:35:13 2022] DELETE /identity/v3/users/8aaeec6b60434c4ba8280111b0eb15d3 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:14.003417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:14.004933 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:14.017615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b78240-4aaa-446e-95e8-8a1637a798af admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:14.089618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fa004-aa41-41d9-af30-403dc428adc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:14.089801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fa004-aa41-41d9-af30-403dc428adc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:14.090045 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fa004-aa41-41d9-af30-403dc428adc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:14.090292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fa004-aa41-41d9-af30-403dc428adc0 admin admin] PATH_INFO: `/v3/projects/e821263aefb749d49f94c08266fc8abf` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:14.091480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fa004-aa41-41d9-af30-403dc428adc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e821263aefb749d49f94c08266fc8abf)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:14.093483 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fa004-aa41-41d9-af30-403dc428adc0 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:14.150433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-121fa004-aa41-41d9-af30-403dc428adc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e821263aefb749d49f94c08266fc8abf', 'request_id': 'req-121fa004-aa41-41d9-af30-403dc428adc0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:14.156413 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-121fa004-aa41-41d9-af30-403dc428adc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e821263aefb749d49f94c08266fc8abf', 'request_id': 'req-121fa004-aa41-41d9-af30-403dc428adc0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:14.165128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-121fa004-aa41-41d9-af30-403dc428adc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e821263aefb749d49f94c08266fc8abf', 'request_id': 'req-121fa004-aa41-41d9-af30-403dc428adc0'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:14.176633 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-121fa004-aa41-41d9-af30-403dc428adc0 admin admin] The token cache is being invalidate because project e821263aefb749d49f94c08266fc8abf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:35:14.176982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-121fa004-aa41-41d9-af30-403dc428adc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:14.177559 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-121fa004-aa41-41d9-af30-403dc428adc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:14.179278 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1364/2734] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:35:14 2022] DELETE /identity/v3/projects/e821263aefb749d49f94c08266fc8abf => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:14.184201 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31d89a-56ed-482b-b319-21893bc108e8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:14.184483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31d89a-56ed-482b-b319-21893bc108e8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:14.184757 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31d89a-56ed-482b-b319-21893bc108e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:14.205113 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8f31d89a-56ed-482b-b319-21893bc108e8 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:14.206962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8f31d89a-56ed-482b-b319-21893bc108e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:14.267015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1371/2735] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:14.275748 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a376d8-6596-4c22-8b53-65f09b177e87 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:14.276036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a376d8-6596-4c22-8b53-65f09b177e87 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:14.276312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a376d8-6596-4c22-8b53-65f09b177e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:14.297029 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-16a376d8-6596-4c22-8b53-65f09b177e87 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:14.298344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-16a376d8-6596-4c22-8b53-65f09b177e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:14.306317 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1365/2736] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:14.310790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31d89a-56ed-482b-b319-21893bc108e8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:14.321541 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31d89a-56ed-482b-b319-21893bc108e8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:14.326336 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37a4ce-b8e1-4295-8f87-2c0f6097f7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:14.327035 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37a4ce-b8e1-4295-8f87-2c0f6097f7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:14.327370 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37a4ce-b8e1-4295-8f87-2c0f6097f7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:14.327673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37a4ce-b8e1-4295-8f87-2c0f6097f7b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:14.328326 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37a4ce-b8e1-4295-8f87-2c0f6097f7b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:35:14.328629 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37a4ce-b8e1-4295-8f87-2c0f6097f7b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:14.330216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37a4ce-b8e1-4295-8f87-2c0f6097f7b1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:14.345601 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1372/2737] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:35:14.347329 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a376d8-6596-4c22-8b53-65f09b177e87 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:14.356830 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a376d8-6596-4c22-8b53-65f09b177e87 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:14.361500 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c537db-47b6-4810-aa16-1cd90d716453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:14.362103 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c537db-47b6-4810-aa16-1cd90d716453 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:14.362948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c537db-47b6-4810-aa16-1cd90d716453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:14.363223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c537db-47b6-4810-aa16-1cd90d716453 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:14.363894 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c537db-47b6-4810-aa16-1cd90d716453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:14.365797 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c537db-47b6-4810-aa16-1cd90d716453 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:14.371217 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:14.403385 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1366/2738] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:35:14 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:14.404922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37a4ce-b8e1-4295-8f87-2c0f6097f7b1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:14.415999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37a4ce-b8e1-4295-8f87-2c0f6097f7b1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:14.497277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7719a0-93de-46d0-b4f9-1edaff7084fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:14.497945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7719a0-93de-46d0-b4f9-1edaff7084fd admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:14.498528 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7719a0-93de-46d0-b4f9-1edaff7084fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:14.498809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7719a0-93de-46d0-b4f9-1edaff7084fd admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:14.499489 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7719a0-93de-46d0-b4f9-1edaff7084fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:14.501452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7719a0-93de-46d0-b4f9-1edaff7084fd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:14.526669 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1373/2739] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:35:14 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:14.532165 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c537db-47b6-4810-aa16-1cd90d716453 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:14.547913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c537db-47b6-4810-aa16-1cd90d716453 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:14.552845 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962489b-73c7-4beb-98c0-a53b71f83da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:14.553454 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962489b-73c7-4beb-98c0-a53b71f83da0 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:14.553752 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962489b-73c7-4beb-98c0-a53b71f83da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:14.554177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962489b-73c7-4beb-98c0-a53b71f83da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:14.554822 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962489b-73c7-4beb-98c0-a53b71f83da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:14.556530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962489b-73c7-4beb-98c0-a53b71f83da0 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:14.567618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1367/2740] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:35:14.570988 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7719a0-93de-46d0-b4f9-1edaff7084fd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:14.583691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7719a0-93de-46d0-b4f9-1edaff7084fd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:14.589558 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3ff13-4fb4-404a-b59a-5ef1ef054a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:14.590215 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3ff13-4fb4-404a-b59a-5ef1ef054a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:14.590497 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3ff13-4fb4-404a-b59a-5ef1ef054a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:14.590804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3ff13-4fb4-404a-b59a-5ef1ef054a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:14.591507 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3ff13-4fb4-404a-b59a-5ef1ef054a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:14.593229 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3ff13-4fb4-404a-b59a-5ef1ef054a9e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:14.608027 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1374/2741] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:35:14.612620 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962489b-73c7-4beb-98c0-a53b71f83da0 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:14.623177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962489b-73c7-4beb-98c0-a53b71f83da0 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:14.629697 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85070dbd-881f-4bc2-95b7-68ad075e81c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:14.630421 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85070dbd-881f-4bc2-95b7-68ad075e81c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:14.630703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85070dbd-881f-4bc2-95b7-68ad075e81c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:14.630973 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85070dbd-881f-4bc2-95b7-68ad075e81c0 admin admin] PATH_INFO: `/v3/projects/ccbaec5fdb0c49cc854822ab9d22ff26/users/a0ae8218b852492aad4a707e72da8b86/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:14.650602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85070dbd-881f-4bc2-95b7-68ad075e81c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccbaec5fdb0c49cc854822ab9d22ff26, user_id=a0ae8218b852492aad4a707e72da8b86, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:14.653450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85070dbd-881f-4bc2-95b7-68ad075e81c0 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:14.674469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1368/2742] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:35:14 2022] PUT /identity/v3/projects/ccbaec5fdb0c49cc854822ab9d22ff26/users/a0ae8218b852492aad4a707e72da8b86/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:35:14.686954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448335bd-8985-4d4d-9101-79302a95172f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:14.687252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448335bd-8985-4d4d-9101-79302a95172f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:14.687591 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448335bd-8985-4d4d-9101-79302a95172f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:14.753790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-448335bd-8985-4d4d-9101-79302a95172f None None] MFA Rules not processed for user `a0ae8218b852492aad4a707e72da8b86`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:14.755311 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-448335bd-8985-4d4d-9101-79302a95172f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:14.821936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:14.866092 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1375/2743] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:14.880300 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85070dbd-881f-4bc2-95b7-68ad075e81c0 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:14.962618 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85070dbd-881f-4bc2-95b7-68ad075e81c0 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:14.967626 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6227e8a-9f45-4909-95e1-0ebf8112969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:14.968174 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6227e8a-9f45-4909-95e1-0ebf8112969c admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:14.968452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6227e8a-9f45-4909-95e1-0ebf8112969c admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:14.968723 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6227e8a-9f45-4909-95e1-0ebf8112969c admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:14.969348 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6227e8a-9f45-4909-95e1-0ebf8112969c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:14.971221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6227e8a-9f45-4909-95e1-0ebf8112969c admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:14.998612 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1369/2744] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:35:14 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:15.002429 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448335bd-8985-4d4d-9101-79302a95172f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:15.013293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448335bd-8985-4d4d-9101-79302a95172f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:15.069205 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609ae00-7a42-4020-aa2c-a3e5a7a992fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:15.070032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609ae00-7a42-4020-aa2c-a3e5a7a992fa admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:15.070493 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609ae00-7a42-4020-aa2c-a3e5a7a992fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:15.070928 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609ae00-7a42-4020-aa2c-a3e5a7a992fa admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:15.071773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609ae00-7a42-4020-aa2c-a3e5a7a992fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:15.073848 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609ae00-7a42-4020-aa2c-a3e5a7a992fa admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:15.100088 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1376/2745] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:35:15 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:15.101310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6227e8a-9f45-4909-95e1-0ebf8112969c admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:15.111796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6227e8a-9f45-4909-95e1-0ebf8112969c admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:15.117043 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe2e77-0e29-4bc0-af27-97d1562bf09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:15.117637 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe2e77-0e29-4bc0-af27-97d1562bf09d admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:15.117921 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe2e77-0e29-4bc0-af27-97d1562bf09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:15.118198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe2e77-0e29-4bc0-af27-97d1562bf09d admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:15.118842 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe2e77-0e29-4bc0-af27-97d1562bf09d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:15.120478 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe2e77-0e29-4bc0-af27-97d1562bf09d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:15.130984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1370/2746] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:35:15.135086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609ae00-7a42-4020-aa2c-a3e5a7a992fa admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:15.145886 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609ae00-7a42-4020-aa2c-a3e5a7a992fa admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:15.150996 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f76374-b6a3-4ee7-9240-b8fb45b59c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:15.151631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f76374-b6a3-4ee7-9240-b8fb45b59c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:15.151964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f76374-b6a3-4ee7-9240-b8fb45b59c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:15.152244 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f76374-b6a3-4ee7-9240-b8fb45b59c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:15.153056 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f76374-b6a3-4ee7-9240-b8fb45b59c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:15.154722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f76374-b6a3-4ee7-9240-b8fb45b59c10 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:15.167616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1377/2747] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:35:15.170824 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe2e77-0e29-4bc0-af27-97d1562bf09d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:15.183163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe2e77-0e29-4bc0-af27-97d1562bf09d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:15.188602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b3db2-fc49-4c02-b751-91d884d1a2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:15.189328 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3db2-fc49-4c02-b751-91d884d1a2a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:15.189880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3db2-fc49-4c02-b751-91d884d1a2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:15.190000 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3db2-fc49-4c02-b751-91d884d1a2a7 admin admin] PATH_INFO: `/v3/projects/978d7c06878949a389f335b150f7dc0b/users/5e603278d4a644f79a881265657571f4/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:15.206443 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b3db2-fc49-4c02-b751-91d884d1a2a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=978d7c06878949a389f335b150f7dc0b, user_id=5e603278d4a644f79a881265657571f4, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:15.209173 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b3db2-fc49-4c02-b751-91d884d1a2a7 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:15.224474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1371/2748] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:35:15 2022] PUT /identity/v3/projects/978d7c06878949a389f335b150f7dc0b/users/5e603278d4a644f79a881265657571f4/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:15.236333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e67c57-fa26-415f-9923-bf9917080ca5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:15.236663 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e67c57-fa26-415f-9923-bf9917080ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:15.236975 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e67c57-fa26-415f-9923-bf9917080ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:15.266458 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-77e67c57-fa26-415f-9923-bf9917080ca5 None None] MFA Rules not processed for user `5e603278d4a644f79a881265657571f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:15.267727 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-77e67c57-fa26-415f-9923-bf9917080ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:15.335339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1378/2749] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:15.340182 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b3db2-fc49-4c02-b751-91d884d1a2a7 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:15.402062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b3db2-fc49-4c02-b751-91d884d1a2a7 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:15.406741 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e36306-a3dd-42d7-9f57-76f17db46426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:15.407345 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e36306-a3dd-42d7-9f57-76f17db46426 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:15.407664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e36306-a3dd-42d7-9f57-76f17db46426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:15.407944 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e36306-a3dd-42d7-9f57-76f17db46426 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:15.408573 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e36306-a3dd-42d7-9f57-76f17db46426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:15.411347 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e36306-a3dd-42d7-9f57-76f17db46426 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:15.417090 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:15.443280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1372/2750] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:35:15 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:35:15.447387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e67c57-fa26-415f-9923-bf9917080ca5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:15.458537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e67c57-fa26-415f-9923-bf9917080ca5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:15.517559 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:15.533269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0df544-94a1-427e-8000-99700eb6a953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:15.533882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0df544-94a1-427e-8000-99700eb6a953 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:15.534722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0df544-94a1-427e-8000-99700eb6a953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:15.534997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0df544-94a1-427e-8000-99700eb6a953 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:15.535660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0df544-94a1-427e-8000-99700eb6a953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:15.537565 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0df544-94a1-427e-8000-99700eb6a953 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:15.558653 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1379/2751] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:35:15 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:15.562488 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e36306-a3dd-42d7-9f57-76f17db46426 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:15.572798 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e36306-a3dd-42d7-9f57-76f17db46426 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:15.577481 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea91fb1-3daf-4b01-ab84-6ca8a4e32df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:15.578074 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea91fb1-3daf-4b01-ab84-6ca8a4e32df2 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:15.578351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea91fb1-3daf-4b01-ab84-6ca8a4e32df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:15.578630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea91fb1-3daf-4b01-ab84-6ca8a4e32df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:15.579260 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea91fb1-3daf-4b01-ab84-6ca8a4e32df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:15.581456 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea91fb1-3daf-4b01-ab84-6ca8a4e32df2 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:15.592569 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1373/2752] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:35:15.597419 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0df544-94a1-427e-8000-99700eb6a953 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:15.608320 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0df544-94a1-427e-8000-99700eb6a953 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:15.613388 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123875dd-c67f-4ffb-a613-8089c342ee7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:15.613642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123875dd-c67f-4ffb-a613-8089c342ee7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:15.613920 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123875dd-c67f-4ffb-a613-8089c342ee7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:15.614225 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123875dd-c67f-4ffb-a613-8089c342ee7b admin admin] PATH_INFO: `/v3/projects/b1adc7ce3ec84c92be67c9f40fc95380/users/0afb73b47c924aa79c212c055dfb789f/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:15.629399 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123875dd-c67f-4ffb-a613-8089c342ee7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1adc7ce3ec84c92be67c9f40fc95380, user_id=0afb73b47c924aa79c212c055dfb789f, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:15.632294 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123875dd-c67f-4ffb-a613-8089c342ee7b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:15.646049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1380/2753] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:35:15 2022] PUT /identity/v3/projects/b1adc7ce3ec84c92be67c9f40fc95380/users/0afb73b47c924aa79c212c055dfb789f/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:35:15.650266 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea91fb1-3daf-4b01-ab84-6ca8a4e32df2 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:15.660400 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea91fb1-3daf-4b01-ab84-6ca8a4e32df2 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:15.719451 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650d834-ed28-4ed2-b868-9380ac9fb22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:15.720185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650d834-ed28-4ed2-b868-9380ac9fb22f admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:15.720553 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650d834-ed28-4ed2-b868-9380ac9fb22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:15.720832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650d834-ed28-4ed2-b868-9380ac9fb22f admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:15.721515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650d834-ed28-4ed2-b868-9380ac9fb22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:15.723179 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650d834-ed28-4ed2-b868-9380ac9fb22f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:15.732958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1374/2754] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:35:15.737784 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123875dd-c67f-4ffb-a613-8089c342ee7b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:15.748338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123875dd-c67f-4ffb-a613-8089c342ee7b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:15.753480 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9c374-bcf4-4fcf-acea-d4e2d511e6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:15.754134 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9c374-bcf4-4fcf-acea-d4e2d511e6bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:15.754418 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9c374-bcf4-4fcf-acea-d4e2d511e6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:15.754721 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9c374-bcf4-4fcf-acea-d4e2d511e6bc admin admin] PATH_INFO: `/v3/projects/b1adc7ce3ec84c92be67c9f40fc95380/users/0afb73b47c924aa79c212c055dfb789f/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:15.757158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9c374-bcf4-4fcf-acea-d4e2d511e6bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1adc7ce3ec84c92be67c9f40fc95380, user_id=0afb73b47c924aa79c212c055dfb789f, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:15.760696 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9c374-bcf4-4fcf-acea-d4e2d511e6bc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:15.771874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1381/2755] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:35:15 2022] PUT /identity/v3/projects/b1adc7ce3ec84c92be67c9f40fc95380/users/0afb73b47c924aa79c212c055dfb789f/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:15.783399 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122952da-a525-4404-878f-15c13c9a04ce None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:15.783720 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122952da-a525-4404-878f-15c13c9a04ce None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:15.784064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122952da-a525-4404-878f-15c13c9a04ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:15.823784 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-122952da-a525-4404-878f-15c13c9a04ce None None] MFA Rules not processed for user `0afb73b47c924aa79c212c055dfb789f`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:15.825030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-122952da-a525-4404-878f-15c13c9a04ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:15.910734 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1375/2756] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:15.927986 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9c374-bcf4-4fcf-acea-d4e2d511e6bc admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:15.929878 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-81d9c374-bcf4-4fcf-acea-d4e2d511e6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:16.005027 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9c374-bcf4-4fcf-acea-d4e2d511e6bc admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:16.009288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67bbff-9021-4537-8206-2a36da62b0e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:16.009962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67bbff-9021-4537-8206-2a36da62b0e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:16.010237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67bbff-9021-4537-8206-2a36da62b0e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:16.010508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67bbff-9021-4537-8206-2a36da62b0e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:16.079184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67bbff-9021-4537-8206-2a36da62b0e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:16.080690 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67bbff-9021-4537-8206-2a36da62b0e0 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:16.092481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1382/2757] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:35:17.660592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122952da-a525-4404-878f-15c13c9a04ce None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:17.662617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-122952da-a525-4404-878f-15c13c9a04ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:17.676297 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122952da-a525-4404-878f-15c13c9a04ce None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:17.679915 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f70673-e010-4d3b-bc9f-3d4c1b83da15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:17.680648 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f70673-e010-4d3b-bc9f-3d4c1b83da15 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:17.680938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f70673-e010-4d3b-bc9f-3d4c1b83da15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:17.681215 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f70673-e010-4d3b-bc9f-3d4c1b83da15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:17.750349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:17.753237 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f70673-e010-4d3b-bc9f-3d4c1b83da15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:17.755892 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f70673-e010-4d3b-bc9f-3d4c1b83da15 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:17.765683 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1376/2758] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:35:18.299092 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67bbff-9021-4537-8206-2a36da62b0e0 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:18.308280 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67bbff-9021-4537-8206-2a36da62b0e0 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:18.313125 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4657a-9f69-48ca-b761-2474e15c73bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:18.313936 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4657a-9f69-48ca-b761-2474e15c73bd service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:18.314416 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4657a-9f69-48ca-b761-2474e15c73bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:18.314863 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4657a-9f69-48ca-b761-2474e15c73bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:18.327199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4657a-9f69-48ca-b761-2474e15c73bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:18.328843 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4657a-9f69-48ca-b761-2474e15c73bd service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:18.340512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1383/2759] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:35:22.245308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f70673-e010-4d3b-bc9f-3d4c1b83da15 service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:22.306379 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f70673-e010-4d3b-bc9f-3d4c1b83da15 service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:22.310910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac5a27-8a2f-46ae-a981-be370c699855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:22.311540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac5a27-8a2f-46ae-a981-be370c699855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:22.311849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac5a27-8a2f-46ae-a981-be370c699855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:22.312122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac5a27-8a2f-46ae-a981-be370c699855 admin admin] PATH_INFO: `/v3/users/a0ae8218b852492aad4a707e72da8b86` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:22.313238 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac5a27-8a2f-46ae-a981-be370c699855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0ae8218b852492aad4a707e72da8b86)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:22.315763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac5a27-8a2f-46ae-a981-be370c699855 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:22.361375 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f0ac5a27-8a2f-46ae-a981-be370c699855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0ae8218b852492aad4a707e72da8b86'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:22.373503 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f0ac5a27-8a2f-46ae-a981-be370c699855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0ae8218b852492aad4a707e72da8b86'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:22.383288 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f0ac5a27-8a2f-46ae-a981-be370c699855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0ae8218b852492aad4a707e72da8b86'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:22.384271 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f0ac5a27-8a2f-46ae-a981-be370c699855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0ae8218b852492aad4a707e72da8b86'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:22.395201 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-f0ac5a27-8a2f-46ae-a981-be370c699855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0ae8218b852492aad4a707e72da8b86'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:22.401773 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1377/2760] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:35:22 2022] DELETE /identity/v3/users/a0ae8218b852492aad4a707e72da8b86 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:22.406191 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617172f-d0d5-4b22-abc6-f018d265aff1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:22.406648 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617172f-d0d5-4b22-abc6-f018d265aff1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:22.406916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617172f-d0d5-4b22-abc6-f018d265aff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:22.428624 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3617172f-d0d5-4b22-abc6-f018d265aff1 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:22.429869 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3617172f-d0d5-4b22-abc6-f018d265aff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:22.472448 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:22.495340 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1384/2761] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:22.506211 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac5a27-8a2f-46ae-a981-be370c699855 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:22.507639 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-f0ac5a27-8a2f-46ae-a981-be370c699855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:22.566142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac5a27-8a2f-46ae-a981-be370c699855 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:22.569791 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56319a90-f1ea-42ce-9819-60901517eee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:22.570398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56319a90-f1ea-42ce-9819-60901517eee5 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:22.570676 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56319a90-f1ea-42ce-9819-60901517eee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:22.570945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56319a90-f1ea-42ce-9819-60901517eee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:22.580396 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56319a90-f1ea-42ce-9819-60901517eee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:22.581902 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56319a90-f1ea-42ce-9819-60901517eee5 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:22.591265 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1378/2762] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:35:22.870334 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617172f-d0d5-4b22-abc6-f018d265aff1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:22.871832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3617172f-d0d5-4b22-abc6-f018d265aff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:22.884352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617172f-d0d5-4b22-abc6-f018d265aff1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:22.890047 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:22.890681 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:22.890965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:22.891237 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] PATH_INFO: `/v3/projects/ccbaec5fdb0c49cc854822ab9d22ff26` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:22.892468 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccbaec5fdb0c49cc854822ab9d22ff26)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:22.894883 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:22.945508 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccbaec5fdb0c49cc854822ab9d22ff26', 'request_id': 'req-98cf8133-120a-40dd-986c-53253d807eac'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:22.950915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccbaec5fdb0c49cc854822ab9d22ff26', 'request_id': 'req-98cf8133-120a-40dd-986c-53253d807eac'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:22.958821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccbaec5fdb0c49cc854822ab9d22ff26', 'request_id': 'req-98cf8133-120a-40dd-986c-53253d807eac'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:22.969953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] The token cache is being invalidate because project ccbaec5fdb0c49cc854822ab9d22ff26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:35:22.970242 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:22.970864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:22.972972 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1385/2763] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:35:22 2022] DELETE /identity/v3/projects/ccbaec5fdb0c49cc854822ab9d22ff26 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:22.977020 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56319a90-f1ea-42ce-9819-60901517eee5 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:22.987217 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56319a90-f1ea-42ce-9819-60901517eee5 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:23.000274 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:23.049893 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315e33f-4dcc-4089-8bfe-01518c1e4bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:23.050579 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315e33f-4dcc-4089-8bfe-01518c1e4bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:23.050864 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315e33f-4dcc-4089-8bfe-01518c1e4bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:23.051140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315e33f-4dcc-4089-8bfe-01518c1e4bd6 admin admin] PATH_INFO: `/v3/users/5e603278d4a644f79a881265657571f4` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:23.052336 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315e33f-4dcc-4089-8bfe-01518c1e4bd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e603278d4a644f79a881265657571f4)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:23.054307 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315e33f-4dcc-4089-8bfe-01518c1e4bd6 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:23.105218 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2315e33f-4dcc-4089-8bfe-01518c1e4bd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e603278d4a644f79a881265657571f4'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.120240 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2315e33f-4dcc-4089-8bfe-01518c1e4bd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e603278d4a644f79a881265657571f4'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.130233 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2315e33f-4dcc-4089-8bfe-01518c1e4bd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e603278d4a644f79a881265657571f4'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.130964 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2315e33f-4dcc-4089-8bfe-01518c1e4bd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e603278d4a644f79a881265657571f4'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.139409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-2315e33f-4dcc-4089-8bfe-01518c1e4bd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e603278d4a644f79a881265657571f4'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.147143 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1379/2764] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:35:22 2022] DELETE /identity/v3/users/5e603278d4a644f79a881265657571f4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:23.418341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:23.419945 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:23.435520 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf8133-120a-40dd-986c-53253d807eac admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:23.494290 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:23.494853 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:23.495224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:23.495388 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] PATH_INFO: `/v3/projects/978d7c06878949a389f335b150f7dc0b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:23.496531 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=978d7c06878949a389f335b150f7dc0b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:23.498490 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:23.548312 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '978d7c06878949a389f335b150f7dc0b', 'request_id': 'req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.554335 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '978d7c06878949a389f335b150f7dc0b', 'request_id': 'req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.562253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '978d7c06878949a389f335b150f7dc0b', 'request_id': 'req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.564408 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:23.577006 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] The token cache is being invalidate because project 978d7c06878949a389f335b150f7dc0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:35:23.577291 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.577942 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.579424 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1386/2765] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:35:23 2022] DELETE /identity/v3/projects/978d7c06878949a389f335b150f7dc0b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:35:23.583504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315e33f-4dcc-4089-8bfe-01518c1e4bd6 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:23.585025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2315e33f-4dcc-4089-8bfe-01518c1e4bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:23.598306 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315e33f-4dcc-4089-8bfe-01518c1e4bd6 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:23.656181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20cadff-20ae-47d4-86aa-31eff2cff15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:23.656805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cadff-20ae-47d4-86aa-31eff2cff15e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:23.657084 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cadff-20ae-47d4-86aa-31eff2cff15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:23.657354 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cadff-20ae-47d4-86aa-31eff2cff15e admin admin] PATH_INFO: `/v3/users/0afb73b47c924aa79c212c055dfb789f` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:23.658461 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20cadff-20ae-47d4-86aa-31eff2cff15e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0afb73b47c924aa79c212c055dfb789f)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:23.660437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20cadff-20ae-47d4-86aa-31eff2cff15e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:23.705665 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:23.712749 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-b20cadff-20ae-47d4-86aa-31eff2cff15e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0afb73b47c924aa79c212c055dfb789f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.727314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-b20cadff-20ae-47d4-86aa-31eff2cff15e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0afb73b47c924aa79c212c055dfb789f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.736322 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-b20cadff-20ae-47d4-86aa-31eff2cff15e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0afb73b47c924aa79c212c055dfb789f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.737829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-b20cadff-20ae-47d4-86aa-31eff2cff15e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0afb73b47c924aa79c212c055dfb789f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.745888 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-b20cadff-20ae-47d4-86aa-31eff2cff15e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0afb73b47c924aa79c212c055dfb789f'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:23.753026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1380/2766] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:35:23 2022] DELETE /identity/v3/users/0afb73b47c924aa79c212c055dfb789f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:24.264258 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:24.265741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:24.285276 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4a9a1-c9f3-4762-87d4-0f6173dcce95 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:24.343333 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0a08e-08ef-492d-a755-a57a82623ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:24.343900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0a08e-08ef-492d-a755-a57a82623ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:24.344181 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0a08e-08ef-492d-a755-a57a82623ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:24.344454 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0a08e-08ef-492d-a755-a57a82623ab5 admin admin] PATH_INFO: `/v3/projects/b1adc7ce3ec84c92be67c9f40fc95380` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:24.345555 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0a08e-08ef-492d-a755-a57a82623ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1adc7ce3ec84c92be67c9f40fc95380)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:24.348115 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0a08e-08ef-492d-a755-a57a82623ab5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:24.395434 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eaf0a08e-08ef-492d-a755-a57a82623ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1adc7ce3ec84c92be67c9f40fc95380', 'request_id': 'req-eaf0a08e-08ef-492d-a755-a57a82623ab5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:24.402901 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eaf0a08e-08ef-492d-a755-a57a82623ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1adc7ce3ec84c92be67c9f40fc95380', 'request_id': 'req-eaf0a08e-08ef-492d-a755-a57a82623ab5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:24.411341 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eaf0a08e-08ef-492d-a755-a57a82623ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1adc7ce3ec84c92be67c9f40fc95380', 'request_id': 'req-eaf0a08e-08ef-492d-a755-a57a82623ab5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:24.425096 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eaf0a08e-08ef-492d-a755-a57a82623ab5 admin admin] The token cache is being invalidate because project b1adc7ce3ec84c92be67c9f40fc95380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:35:24.425641 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eaf0a08e-08ef-492d-a755-a57a82623ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:24.426249 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eaf0a08e-08ef-492d-a755-a57a82623ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:35:24.431307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1387/2767] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:35:24 2022] DELETE /identity/v3/projects/b1adc7ce3ec84c92be67c9f40fc95380 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:35:24.433655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba037127-615a-4baf-a0d3-11eea4c8535f None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:24.433954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba037127-615a-4baf-a0d3-11eea4c8535f None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:24.434232 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba037127-615a-4baf-a0d3-11eea4c8535f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:24.457849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-ba037127-615a-4baf-a0d3-11eea4c8535f None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:24.459070 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ba037127-615a-4baf-a0d3-11eea4c8535f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:24.524866 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1381/2768] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:35:24.532922 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dfeab-7900-4e46-94e6-2e0ffe8724e4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:24.533211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dfeab-7900-4e46-94e6-2e0ffe8724e4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:24.533537 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dfeab-7900-4e46-94e6-2e0ffe8724e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:24.553970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-523dfeab-7900-4e46-94e6-2e0ffe8724e4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:24.555192 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-523dfeab-7900-4e46-94e6-2e0ffe8724e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:24.563379 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1388/2769] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:24.567632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba037127-615a-4baf-a0d3-11eea4c8535f None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:24.577143 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba037127-615a-4baf-a0d3-11eea4c8535f None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:24.582435 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39ef66-5c6c-4edd-bdee-64c72a21e6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:24.583039 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39ef66-5c6c-4edd-bdee-64c72a21e6bb admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:24.583323 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39ef66-5c6c-4edd-bdee-64c72a21e6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:24.583622 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39ef66-5c6c-4edd-bdee-64c72a21e6bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:24.584276 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39ef66-5c6c-4edd-bdee-64c72a21e6bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:35:24.584582 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39ef66-5c6c-4edd-bdee-64c72a21e6bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:24.586194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39ef66-5c6c-4edd-bdee-64c72a21e6bb admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:24.598419 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1382/2770] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:35:24.603086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523dfeab-7900-4e46-94e6-2e0ffe8724e4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:24.612382 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523dfeab-7900-4e46-94e6-2e0ffe8724e4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:24.617175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9db73-68f8-435f-a490-63c3a995fdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:24.617777 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9db73-68f8-435f-a490-63c3a995fdb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:24.618055 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9db73-68f8-435f-a490-63c3a995fdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:24.618328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9db73-68f8-435f-a490-63c3a995fdb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:24.618953 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9db73-68f8-435f-a490-63c3a995fdb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:24.620880 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9db73-68f8-435f-a490-63c3a995fdb6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:24.649808 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1389/2771] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:35:24 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:24.654194 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39ef66-5c6c-4edd-bdee-64c72a21e6bb admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:24.664029 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39ef66-5c6c-4edd-bdee-64c72a21e6bb admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:24.725908 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd9a3f-c29e-4a96-be95-0d2332e4186d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:24.726515 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd9a3f-c29e-4a96-be95-0d2332e4186d admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:24.726798 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd9a3f-c29e-4a96-be95-0d2332e4186d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:24.727072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd9a3f-c29e-4a96-be95-0d2332e4186d admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:24.727743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd9a3f-c29e-4a96-be95-0d2332e4186d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:24.729726 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd9a3f-c29e-4a96-be95-0d2332e4186d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:24.753950 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1383/2772] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:35:24 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:35:24.758064 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9db73-68f8-435f-a490-63c3a995fdb6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:24.769128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9db73-68f8-435f-a490-63c3a995fdb6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:24.775077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4e3a2-ddf1-45b5-910b-a705aa2ea08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:24.776078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4e3a2-ddf1-45b5-910b-a705aa2ea08d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:24.776078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4e3a2-ddf1-45b5-910b-a705aa2ea08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:24.776361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4e3a2-ddf1-45b5-910b-a705aa2ea08d admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:24.777007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4e3a2-ddf1-45b5-910b-a705aa2ea08d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:24.778625 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4e3a2-ddf1-45b5-910b-a705aa2ea08d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:24.789634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1390/2773] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:35:24.794005 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd9a3f-c29e-4a96-be95-0d2332e4186d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:24.815286 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd9a3f-c29e-4a96-be95-0d2332e4186d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:24.822143 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac496f27-e15e-4caf-995b-5184a556f231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:24.823107 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496f27-e15e-4caf-995b-5184a556f231 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:24.823632 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496f27-e15e-4caf-995b-5184a556f231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:24.824688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496f27-e15e-4caf-995b-5184a556f231 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:24.825605 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac496f27-e15e-4caf-995b-5184a556f231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:24.827437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac496f27-e15e-4caf-995b-5184a556f231 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:24.848131 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1384/2774] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:35:24.849007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4e3a2-ddf1-45b5-910b-a705aa2ea08d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:24.865980 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4e3a2-ddf1-45b5-910b-a705aa2ea08d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:24.878133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bf59f-8429-4c1b-b44f-63eb2205e33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:24.878874 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bf59f-8429-4c1b-b44f-63eb2205e33f admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:24.879158 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bf59f-8429-4c1b-b44f-63eb2205e33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:24.879490 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bf59f-8429-4c1b-b44f-63eb2205e33f admin admin] PATH_INFO: `/v3/projects/24ded699e3a3432cbe1656777da8f958/users/b72692bbe1e9455db47a1b7e5995c01c/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:24.904927 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bf59f-8429-4c1b-b44f-63eb2205e33f admin admin] RBAC: Authorizing `identity:create_grant(project_id=24ded699e3a3432cbe1656777da8f958, user_id=b72692bbe1e9455db47a1b7e5995c01c, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:24.908075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bf59f-8429-4c1b-b44f-63eb2205e33f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:24.935606 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1391/2775] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:35:24 2022] PUT /identity/v3/projects/24ded699e3a3432cbe1656777da8f958/users/b72692bbe1e9455db47a1b7e5995c01c/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:35:24.947510 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2370a-d6ba-4cac-b009-3d69d49a0eac None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:24.948137 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2370a-d6ba-4cac-b009-3d69d49a0eac None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:24.948520 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2370a-d6ba-4cac-b009-3d69d49a0eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:25.004000 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-bcc2370a-d6ba-4cac-b009-3d69d49a0eac None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:25.008136 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bcc2370a-d6ba-4cac-b009-3d69d49a0eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:25.029401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:25.086880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1385/2776] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:25.108445 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bf59f-8429-4c1b-b44f-63eb2205e33f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:25.110658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-412bf59f-8429-4c1b-b44f-63eb2205e33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:25.142754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:25.186291 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bf59f-8429-4c1b-b44f-63eb2205e33f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:25.191188 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce689c2c-da1d-4950-b74e-357c381ff748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:25.191929 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce689c2c-da1d-4950-b74e-357c381ff748 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:25.192202 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce689c2c-da1d-4950-b74e-357c381ff748 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:25.192919 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce689c2c-da1d-4950-b74e-357c381ff748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:25.202855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce689c2c-da1d-4950-b74e-357c381ff748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:25.204584 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce689c2c-da1d-4950-b74e-357c381ff748 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:25.217253 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1392/2777] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:35:25.553296 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2370a-d6ba-4cac-b009-3d69d49a0eac None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:25.554942 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bcc2370a-d6ba-4cac-b009-3d69d49a0eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:25.567863 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2370a-d6ba-4cac-b009-3d69d49a0eac None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:25.571848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddadd08-12a8-4170-98ad-10dd883779b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:25.572464 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddadd08-12a8-4170-98ad-10dd883779b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:25.573164 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddadd08-12a8-4170-98ad-10dd883779b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:25.573448 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddadd08-12a8-4170-98ad-10dd883779b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:25.575326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-eddadd08-12a8-4170-98ad-10dd883779b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:25.645849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddadd08-12a8-4170-98ad-10dd883779b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:25.647308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddadd08-12a8-4170-98ad-10dd883779b3 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:25.657947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1386/2778] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:35:26.520517 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35faac04-3868-4306-960d-c448dd3c26f9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:26.521567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35faac04-3868-4306-960d-c448dd3c26f9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:26.521947 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35faac04-3868-4306-960d-c448dd3c26f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:26.551613 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-35faac04-3868-4306-960d-c448dd3c26f9 None None] MFA Rules not processed for user `b72692bbe1e9455db47a1b7e5995c01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:26.552828 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-35faac04-3868-4306-960d-c448dd3c26f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:26.625570 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1393/2779] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:26.634596 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddadd08-12a8-4170-98ad-10dd883779b3 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:26.636228 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-eddadd08-12a8-4170-98ad-10dd883779b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:26.699064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddadd08-12a8-4170-98ad-10dd883779b3 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:26.704323 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfcd78-e657-4bf2-b2d2-3cbf068184dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:26.705043 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfcd78-e657-4bf2-b2d2-3cbf068184dd service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:26.705452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfcd78-e657-4bf2-b2d2-3cbf068184dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:26.705890 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfcd78-e657-4bf2-b2d2-3cbf068184dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:26.715154 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfcd78-e657-4bf2-b2d2-3cbf068184dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:26.716646 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfcd78-e657-4bf2-b2d2-3cbf068184dd service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:26.726847 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1387/2780] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:35:28.136148 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f54d95-8d5a-4480-a3a2-7a83020de76b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:28.136733 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f54d95-8d5a-4480-a3a2-7a83020de76b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:28.136957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f54d95-8d5a-4480-a3a2-7a83020de76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:28.159735 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f9f54d95-8d5a-4480-a3a2-7a83020de76b None None] MFA Rules not processed for user `b72692bbe1e9455db47a1b7e5995c01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:28.161264 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f9f54d95-8d5a-4480-a3a2-7a83020de76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:28.172784 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1394/2781] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:36.722326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=73 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:38.154027 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=51 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:35:46.799740 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfcd78-e657-4bf2-b2d2-3cbf068184dd service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:35:46.801281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-47bfcd78-e657-4bf2-b2d2-3cbf068184dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:46.813077 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfcd78-e657-4bf2-b2d2-3cbf068184dd service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:35:46.817191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f7ed8-3391-4740-a59f-045d35fc102b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:35:46.817784 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f7ed8-3391-4740-a59f-045d35fc102b service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:46.818060 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f7ed8-3391-4740-a59f-045d35fc102b service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:46.818327 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f7ed8-3391-4740-a59f-045d35fc102b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:46.820143 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b17f7ed8-3391-4740-a59f-045d35fc102b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:46.879056 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f7ed8-3391-4740-a59f-045d35fc102b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:35:46.880567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f7ed8-3391-4740-a59f-045d35fc102b service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:35:46.889516 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1388/2782] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:35:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:35:54.787758 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36a9f0-fe6f-4866-9197-78fd0f3d8edf None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:35:54.788250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36a9f0-fe6f-4866-9197-78fd0f3d8edf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:35:54.788576 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36a9f0-fe6f-4866-9197-78fd0f3d8edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:35:54.809567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3c36a9f0-fe6f-4866-9197-78fd0f3d8edf None None] MFA Rules not processed for user `2d62232282dc4455834857d5290eb169`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:35:54.810790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3c36a9f0-fe6f-4866-9197-78fd0f3d8edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:35:54.819452 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1395/2783] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:35:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:35:56.885344 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=28 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:36:04.212694 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f7ed8-3391-4740-a59f-045d35fc102b service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:36:04.214997 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b17f7ed8-3391-4740-a59f-045d35fc102b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:36:04.403337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f7ed8-3391-4740-a59f-045d35fc102b service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:36:04.408381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ee354-979f-4066-950a-0daea6d8a9df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b3b0a0317b14e29a32eb6ba6b89c2aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:36:04.408981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ee354-979f-4066-950a-0daea6d8a9df service placement] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:36:04.409259 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ee354-979f-4066-950a-0daea6d8a9df service placement] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:36:04.409529 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ee354-979f-4066-950a-0daea6d8a9df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:36:04.411292 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-0e1ee354-979f-4066-950a-0daea6d8a9df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:36:04.428576 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ee354-979f-4066-950a-0daea6d8a9df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:36:04.430338 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ee354-979f-4066-950a-0daea6d8a9df service placement] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:36:04.441707 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1389/2784] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:36:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:36:04.803421 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:36:07.807815 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c36a9f0-fe6f-4866-9197-78fd0f3d8edf None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:36:07.809343 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3c36a9f0-fe6f-4866-9197-78fd0f3d8edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:36:07.890440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c36a9f0-fe6f-4866-9197-78fd0f3d8edf None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:36:07.895765 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bdfc9-0e79-4fa8-be56-dfb576a5a571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:36:07.896391 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bdfc9-0e79-4fa8-be56-dfb576a5a571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:36:07.896670 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bdfc9-0e79-4fa8-be56-dfb576a5a571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:36:07.896939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bdfc9-0e79-4fa8-be56-dfb576a5a571 admin admin] PATH_INFO: `/v3/users/2e22d2226f77454fb3f155dbf3fdf471` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:36:07.898309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bdfc9-0e79-4fa8-be56-dfb576a5a571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e22d2226f77454fb3f155dbf3fdf471)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:36:07.900267 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bdfc9-0e79-4fa8-be56-dfb576a5a571 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:36:07.947111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4d3bdfc9-0e79-4fa8-be56-dfb576a5a571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e22d2226f77454fb3f155dbf3fdf471'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:07.953850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4d3bdfc9-0e79-4fa8-be56-dfb576a5a571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e22d2226f77454fb3f155dbf3fdf471'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:07.966164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4d3bdfc9-0e79-4fa8-be56-dfb576a5a571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e22d2226f77454fb3f155dbf3fdf471'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:07.973338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4d3bdfc9-0e79-4fa8-be56-dfb576a5a571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e22d2226f77454fb3f155dbf3fdf471'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:07.973974 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4d3bdfc9-0e79-4fa8-be56-dfb576a5a571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e22d2226f77454fb3f155dbf3fdf471'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:07.979716 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1396/2785] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:36:07 2022] DELETE /identity/v3/users/2e22d2226f77454fb3f155dbf3fdf471 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:36:07.989345 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ee354-979f-4066-950a-0daea6d8a9df service placement] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:36:07.998685 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ee354-979f-4066-950a-0daea6d8a9df service placement] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:36:08.048070 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6dc41-7027-4eae-b9f5-a08620a1bfe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:36:08.048719 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6dc41-7027-4eae-b9f5-a08620a1bfe9 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:36:08.049025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6dc41-7027-4eae-b9f5-a08620a1bfe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:36:08.049389 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6dc41-7027-4eae-b9f5-a08620a1bfe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:36:08.051808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-4ba6dc41-7027-4eae-b9f5-a08620a1bfe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:36:08.087887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=100 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:36:08.119044 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6dc41-7027-4eae-b9f5-a08620a1bfe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:36:08.120601 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6dc41-7027-4eae-b9f5-a08620a1bfe9 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:36:08.131504 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1390/2786] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:36:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:36:08.303727 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bdfc9-0e79-4fa8-be56-dfb576a5a571 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:36:08.305210 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4d3bdfc9-0e79-4fa8-be56-dfb576a5a571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:36:08.319224 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bdfc9-0e79-4fa8-be56-dfb576a5a571 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:36:08.323831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:36:08.324462 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:36:08.325146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:36:08.325146 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] PATH_INFO: `/v3/projects/e3ac3aed7b1e495f81dc29d224be1eba` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:36:08.326282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3ac3aed7b1e495f81dc29d224be1eba)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:36:08.328219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:36:08.375542 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3ac3aed7b1e495f81dc29d224be1eba', 'request_id': 'req-1fb6bce1-823b-422d-b889-5e13ef2aad11'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.381794 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3ac3aed7b1e495f81dc29d224be1eba', 'request_id': 'req-1fb6bce1-823b-422d-b889-5e13ef2aad11'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.388884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3ac3aed7b1e495f81dc29d224be1eba', 'request_id': 'req-1fb6bce1-823b-422d-b889-5e13ef2aad11'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.396640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:36:08.401032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] The token cache is being invalidate because project e3ac3aed7b1e495f81dc29d224be1eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:36:08.401327 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.402223 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.404044 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1397/2787] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:36:08 2022] DELETE /identity/v3/projects/e3ac3aed7b1e495f81dc29d224be1eba => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:36:08.408136 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6dc41-7027-4eae-b9f5-a08620a1bfe9 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:36:08.418806 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6dc41-7027-4eae-b9f5-a08620a1bfe9 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:36:08.476644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fa31c-d1dd-49f7-9d61-3841a85aa336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:36:08.477282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fa31c-d1dd-49f7-9d61-3841a85aa336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:36:08.477663 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fa31c-d1dd-49f7-9d61-3841a85aa336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:36:08.478081 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fa31c-d1dd-49f7-9d61-3841a85aa336 admin admin] PATH_INFO: `/v3/users/796ccf5ac7fc429db3d181208c72f1c9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:36:08.479259 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fa31c-d1dd-49f7-9d61-3841a85aa336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=796ccf5ac7fc429db3d181208c72f1c9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:36:08.481321 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fa31c-d1dd-49f7-9d61-3841a85aa336 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:36:08.530868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-eb3fa31c-d1dd-49f7-9d61-3841a85aa336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '796ccf5ac7fc429db3d181208c72f1c9'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.543624 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-eb3fa31c-d1dd-49f7-9d61-3841a85aa336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '796ccf5ac7fc429db3d181208c72f1c9'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.551226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-eb3fa31c-d1dd-49f7-9d61-3841a85aa336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '796ccf5ac7fc429db3d181208c72f1c9'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.551935 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-eb3fa31c-d1dd-49f7-9d61-3841a85aa336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '796ccf5ac7fc429db3d181208c72f1c9'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.559157 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-eb3fa31c-d1dd-49f7-9d61-3841a85aa336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '796ccf5ac7fc429db3d181208c72f1c9'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.565333 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1391/2788] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:36:08 2022] DELETE /identity/v3/users/796ccf5ac7fc429db3d181208c72f1c9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:36:08.736370 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:36:08.737840 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:36:08.752908 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6bce1-823b-422d-b889-5e13ef2aad11 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:36:08.811317 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e398c-d981-4973-819f-25bdcecadde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:36:08.812124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e398c-d981-4973-819f-25bdcecadde1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:36:08.812266 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e398c-d981-4973-819f-25bdcecadde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:36:08.812511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e398c-d981-4973-819f-25bdcecadde1 admin admin] PATH_INFO: `/v3/projects/13f4b2186b7342c28a9b62c740750db6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:36:08.813821 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e398c-d981-4973-819f-25bdcecadde1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f4b2186b7342c28a9b62c740750db6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:36:08.815748 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e398c-d981-4973-819f-25bdcecadde1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:36:08.863894 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-af3e398c-d981-4973-819f-25bdcecadde1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f4b2186b7342c28a9b62c740750db6', 'request_id': 'req-af3e398c-d981-4973-819f-25bdcecadde1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.869359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-af3e398c-d981-4973-819f-25bdcecadde1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f4b2186b7342c28a9b62c740750db6', 'request_id': 'req-af3e398c-d981-4973-819f-25bdcecadde1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.876217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-af3e398c-d981-4973-819f-25bdcecadde1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f4b2186b7342c28a9b62c740750db6', 'request_id': 'req-af3e398c-d981-4973-819f-25bdcecadde1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.887706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-af3e398c-d981-4973-819f-25bdcecadde1 admin admin] The token cache is being invalidate because project 13f4b2186b7342c28a9b62c740750db6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:36:08.887998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-af3e398c-d981-4973-819f-25bdcecadde1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.888634 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-af3e398c-d981-4973-819f-25bdcecadde1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:36:08.890153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1398/2789] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:36:08 2022] DELETE /identity/v3/projects/13f4b2186b7342c28a9b62c740750db6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:36:17.220785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fa31c-d1dd-49f7-9d61-3841a85aa336 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:36:17.222325 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-eb3fa31c-d1dd-49f7-9d61-3841a85aa336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:36:17.274022 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:36:17.293394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fa31c-d1dd-49f7-9d61-3841a85aa336 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:36:17.478217 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc9de6-ee2e-4777-97b6-a85abb3fcdda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b3b0a0317b14e29a32eb6ba6b89c2aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:36:17.478830 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc9de6-ee2e-4777-97b6-a85abb3fcdda service placement] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:36:17.479113 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc9de6-ee2e-4777-97b6-a85abb3fcdda service placement] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:36:17.479407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc9de6-ee2e-4777-97b6-a85abb3fcdda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:36:17.481241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-43cc9de6-ee2e-4777-97b6-a85abb3fcdda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:36:17.494104 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc9de6-ee2e-4777-97b6-a85abb3fcdda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:36:17.495811 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc9de6-ee2e-4777-97b6-a85abb3fcdda service placement] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:36:17.509214 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1392/2790] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:36:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:36:18.884695 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:36:25.668328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888a3b9-c652-453b-acb7-419668941825 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:36:25.668955 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888a3b9-c652-453b-acb7-419668941825 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:36:25.668955 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888a3b9-c652-453b-acb7-419668941825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:36:25.687369 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2888a3b9-c652-453b-acb7-419668941825 None None] MFA Rules not processed for user `81501360f082464191731f57d2b6a402`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:36:25.688593 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2888a3b9-c652-453b-acb7-419668941825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:36:25.744657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1399/2791] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:36:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:36:27.502562 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=35 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:36:35.737419 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=25 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:36:38.683757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca17e4-a883-4501-9efe-7180c6a8cf71 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:36:38.684272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca17e4-a883-4501-9efe-7180c6a8cf71 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:36:38.684382 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca17e4-a883-4501-9efe-7180c6a8cf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:36:38.704106 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-25ca17e4-a883-4501-9efe-7180c6a8cf71 None None] MFA Rules not processed for user `b72692bbe1e9455db47a1b7e5995c01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:36:38.705771 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-25ca17e4-a883-4501-9efe-7180c6a8cf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:36:38.761595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1393/2792] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:36:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:36:48.754254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=25 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:06.397647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aba63-2e68-46e8-b10e-22d72db18df2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:06.398174 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aba63-2e68-46e8-b10e-22d72db18df2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:06.398174 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aba63-2e68-46e8-b10e-22d72db18df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:06.428900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-241aba63-2e68-46e8-b10e-22d72db18df2 None None] MFA Rules not processed for user `0212eb5841a04075a0fd7ba853231267`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:06.430203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-241aba63-2e68-46e8-b10e-22d72db18df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:06.493623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1400/2793] 10.222.0.27 () {58 vars in 967 bytes} [Mon Sep 12 23:37:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:37:06.533863 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca17e4-a883-4501-9efe-7180c6a8cf71 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:06.535464 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-25ca17e4-a883-4501-9efe-7180c6a8cf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:06.603486 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca17e4-a883-4501-9efe-7180c6a8cf71 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:06.608362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3fb057-3048-4150-88d0-07b4fa28988b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f6bb09fd7f4905a1693ef7cb6f088e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:06.608958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fb057-3048-4150-88d0-07b4fa28988b service swift] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:06.609236 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fb057-3048-4150-88d0-07b4fa28988b service swift] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:06.609506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3fb057-3048-4150-88d0-07b4fa28988b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:06.617295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3fb057-3048-4150-88d0-07b4fa28988b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:06.618861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3fb057-3048-4150-88d0-07b4fa28988b service swift] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:06.626737 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1394/2794] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Sep 12 23:37:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:37:13.829386 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241aba63-2e68-46e8-b10e-22d72db18df2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:13.831754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-241aba63-2e68-46e8-b10e-22d72db18df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:13.903309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241aba63-2e68-46e8-b10e-22d72db18df2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:13.908476 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71b7fe-6dfa-411a-8397-9b87c8f7564a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:13.909163 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71b7fe-6dfa-411a-8397-9b87c8f7564a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:13.909448 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71b7fe-6dfa-411a-8397-9b87c8f7564a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:13.909725 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71b7fe-6dfa-411a-8397-9b87c8f7564a admin admin] PATH_INFO: `/v3/users/dd6241594ed040ce85c4b017cc361894` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:13.910872 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71b7fe-6dfa-411a-8397-9b87c8f7564a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd6241594ed040ce85c4b017cc361894)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:13.912841 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71b7fe-6dfa-411a-8397-9b87c8f7564a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:13.965773 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9a71b7fe-6dfa-411a-8397-9b87c8f7564a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd6241594ed040ce85c4b017cc361894'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:13.975850 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9a71b7fe-6dfa-411a-8397-9b87c8f7564a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd6241594ed040ce85c4b017cc361894'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:13.991946 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9a71b7fe-6dfa-411a-8397-9b87c8f7564a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd6241594ed040ce85c4b017cc361894'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.001190 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9a71b7fe-6dfa-411a-8397-9b87c8f7564a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd6241594ed040ce85c4b017cc361894'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.002009 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9a71b7fe-6dfa-411a-8397-9b87c8f7564a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd6241594ed040ce85c4b017cc361894'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.013918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1401/2795] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:37:13 2022] DELETE /identity/v3/users/dd6241594ed040ce85c4b017cc361894 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:37:14.213316 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3fb057-3048-4150-88d0-07b4fa28988b service swift] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:14.235472 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3fb057-3048-4150-88d0-07b4fa28988b service swift] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:14.295221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:14.295880 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:14.296156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:14.296426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] PATH_INFO: `/v3/projects/00950b4875f54197bf44bd1ffa353937` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:14.297606 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00950b4875f54197bf44bd1ffa353937)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:14.299533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:14.346240 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00950b4875f54197bf44bd1ffa353937', 'request_id': 'req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.351879 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00950b4875f54197bf44bd1ffa353937', 'request_id': 'req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.358938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00950b4875f54197bf44bd1ffa353937', 'request_id': 'req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.370971 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] The token cache is being invalidate because project 00950b4875f54197bf44bd1ffa353937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:37:14.371258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.371940 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.373433 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1395/2796] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:37:14 2022] DELETE /identity/v3/projects/00950b4875f54197bf44bd1ffa353937 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:37:14.377931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71b7fe-6dfa-411a-8397-9b87c8f7564a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:14.379345 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9a71b7fe-6dfa-411a-8397-9b87c8f7564a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:14.392314 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71b7fe-6dfa-411a-8397-9b87c8f7564a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:14.398737 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:14.454993 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcca8a4-0af3-449f-ab74-a17ae5bbfee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:14.455839 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcca8a4-0af3-449f-ab74-a17ae5bbfee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:14.456125 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcca8a4-0af3-449f-ab74-a17ae5bbfee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:14.456405 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcca8a4-0af3-449f-ab74-a17ae5bbfee6 admin admin] PATH_INFO: `/v3/users/d587495071784b1da16a0d771b5bfc9e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:14.457538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcca8a4-0af3-449f-ab74-a17ae5bbfee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d587495071784b1da16a0d771b5bfc9e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:14.459881 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcca8a4-0af3-449f-ab74-a17ae5bbfee6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:14.515164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bcca8a4-0af3-449f-ab74-a17ae5bbfee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd587495071784b1da16a0d771b5bfc9e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.524205 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bcca8a4-0af3-449f-ab74-a17ae5bbfee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd587495071784b1da16a0d771b5bfc9e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.539339 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bcca8a4-0af3-449f-ab74-a17ae5bbfee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd587495071784b1da16a0d771b5bfc9e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.550644 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bcca8a4-0af3-449f-ab74-a17ae5bbfee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd587495071784b1da16a0d771b5bfc9e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.551316 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3bcca8a4-0af3-449f-ab74-a17ae5bbfee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd587495071784b1da16a0d771b5bfc9e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.563941 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1402/2797] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:37:14 2022] DELETE /identity/v3/users/d587495071784b1da16a0d771b5bfc9e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:37:14.745999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:14.747423 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:14.760636 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc3d33-5626-4545-b413-cbf3b14ca7f8 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:14.777766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:14.818986 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:14.819527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:14.819835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:14.820104 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] PATH_INFO: `/v3/projects/87e1811735e0433e88857ab3b3726533` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:14.822028 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87e1811735e0433e88857ab3b3726533)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:14.823959 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:14.868159 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87e1811735e0433e88857ab3b3726533', 'request_id': 'req-685adcdb-1d74-4c14-916c-4ce03943df44'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.873465 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87e1811735e0433e88857ab3b3726533', 'request_id': 'req-685adcdb-1d74-4c14-916c-4ce03943df44'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.880347 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87e1811735e0433e88857ab3b3726533', 'request_id': 'req-685adcdb-1d74-4c14-916c-4ce03943df44'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.891319 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] The token cache is being invalidate because project 87e1811735e0433e88857ab3b3726533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:37:14.891634 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.892278 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:14.893732 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1396/2798] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:37:14 2022] DELETE /identity/v3/projects/87e1811735e0433e88857ab3b3726533 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:37:24.555075 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:24.888518 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:37.619132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcca8a4-0af3-449f-ab74-a17ae5bbfee6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:37.620637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3bcca8a4-0af3-449f-ab74-a17ae5bbfee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:37.634673 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcca8a4-0af3-449f-ab74-a17ae5bbfee6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:37.699113 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cbf1e9-5bdd-4afd-9a10-8a32de6686a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:37.699785 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbf1e9-5bdd-4afd-9a10-8a32de6686a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:37.700071 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbf1e9-5bdd-4afd-9a10-8a32de6686a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:37.700345 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbf1e9-5bdd-4afd-9a10-8a32de6686a4 admin admin] PATH_INFO: `/v3/users/b72692bbe1e9455db47a1b7e5995c01c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:37.701540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cbf1e9-5bdd-4afd-9a10-8a32de6686a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b72692bbe1e9455db47a1b7e5995c01c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:37.703481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cbf1e9-5bdd-4afd-9a10-8a32de6686a4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:37.752702 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d2cbf1e9-5bdd-4afd-9a10-8a32de6686a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b72692bbe1e9455db47a1b7e5995c01c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:37.759834 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d2cbf1e9-5bdd-4afd-9a10-8a32de6686a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b72692bbe1e9455db47a1b7e5995c01c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:37.772554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d2cbf1e9-5bdd-4afd-9a10-8a32de6686a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b72692bbe1e9455db47a1b7e5995c01c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:37.780855 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d2cbf1e9-5bdd-4afd-9a10-8a32de6686a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b72692bbe1e9455db47a1b7e5995c01c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:37.781652 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d2cbf1e9-5bdd-4afd-9a10-8a32de6686a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b72692bbe1e9455db47a1b7e5995c01c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:37.788496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1403/2799] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:37:37 2022] DELETE /identity/v3/users/b72692bbe1e9455db47a1b7e5995c01c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:37:37.967026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:37.968547 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:37.996351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685adcdb-1d74-4c14-916c-4ce03943df44 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:38.054627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:38.055442 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:38.055842 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:38.056210 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] PATH_INFO: `/v3/projects/24ded699e3a3432cbe1656777da8f958` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:38.057404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=24ded699e3a3432cbe1656777da8f958)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:38.059393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:38.107821 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24ded699e3a3432cbe1656777da8f958', 'request_id': 'req-8b178365-9141-4e59-a818-f2f83331e8fe'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:38.113959 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24ded699e3a3432cbe1656777da8f958', 'request_id': 'req-8b178365-9141-4e59-a818-f2f83331e8fe'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:38.122535 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24ded699e3a3432cbe1656777da8f958', 'request_id': 'req-8b178365-9141-4e59-a818-f2f83331e8fe'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:38.133758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] The token cache is being invalidate because project 24ded699e3a3432cbe1656777da8f958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:37:38.134102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:38.134814 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:37:38.136374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1397/2800] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:37:37 2022] DELETE /identity/v3/projects/24ded699e3a3432cbe1656777da8f958 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:37:38.183151 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb1b88-410d-476c-b8e0-df925b2e6df2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:38.183609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb1b88-410d-476c-b8e0-df925b2e6df2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:38.183970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb1b88-410d-476c-b8e0-df925b2e6df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:38.204713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-09bb1b88-410d-476c-b8e0-df925b2e6df2 None None] MFA Rules not processed for user `81501360f082464191731f57d2b6a402`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:38.205985 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-09bb1b88-410d-476c-b8e0-df925b2e6df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:38.269116 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1404/2801] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:37:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:37:48.130836 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:48.262175 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:13.399793 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:13.401200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:13.479249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b178365-9141-4e59-a818-f2f83331e8fe admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:13.483025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4e693-3742-424b-a562-f40f630aea57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:13.483666 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4e693-3742-424b-a562-f40f630aea57 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:13.483948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4e693-3742-424b-a562-f40f630aea57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:13.484287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4e693-3742-424b-a562-f40f630aea57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:13.486548 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-31d4e693-3742-424b-a562-f40f630aea57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:13.565884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4e693-3742-424b-a562-f40f630aea57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:13.567118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4e693-3742-424b-a562-f40f630aea57 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:13.589426 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1398/2802] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:38:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:38:16.530313 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb1b88-410d-476c-b8e0-df925b2e6df2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:16.540538 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb1b88-410d-476c-b8e0-df925b2e6df2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:16.544086 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4adf3-0a68-472d-bfa1-cee1e2359091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:16.544679 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4adf3-0a68-472d-bfa1-cee1e2359091 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:16.544991 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4adf3-0a68-472d-bfa1-cee1e2359091 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:16.545211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4adf3-0a68-472d-bfa1-cee1e2359091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:16.546898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f3c4adf3-0a68-472d-bfa1-cee1e2359091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:16.608901 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4adf3-0a68-472d-bfa1-cee1e2359091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:16.610900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4adf3-0a68-472d-bfa1-cee1e2359091 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:16.620470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1405/2803] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:38:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:38:22.863871 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4e693-3742-424b-a562-f40f630aea57 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:22.865339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-31d4e693-3742-424b-a562-f40f630aea57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:22.879360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4e693-3742-424b-a562-f40f630aea57 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:22.883958 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce0cbf-02f4-49a1-953b-032c85fdf59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:22.884609 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce0cbf-02f4-49a1-953b-032c85fdf59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:22.884914 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce0cbf-02f4-49a1-953b-032c85fdf59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:22.885192 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce0cbf-02f4-49a1-953b-032c85fdf59e admin admin] PATH_INFO: `/v3/users/2d62232282dc4455834857d5290eb169` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:22.886304 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce0cbf-02f4-49a1-953b-032c85fdf59e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d62232282dc4455834857d5290eb169)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:22.888253 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce0cbf-02f4-49a1-953b-032c85fdf59e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:22.939945 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-a2ce0cbf-02f4-49a1-953b-032c85fdf59e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d62232282dc4455834857d5290eb169'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:22.946887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:22.952865 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-a2ce0cbf-02f4-49a1-953b-032c85fdf59e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d62232282dc4455834857d5290eb169'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:22.959863 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-a2ce0cbf-02f4-49a1-953b-032c85fdf59e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d62232282dc4455834857d5290eb169'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:22.960482 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-a2ce0cbf-02f4-49a1-953b-032c85fdf59e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d62232282dc4455834857d5290eb169'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:22.967357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-a2ce0cbf-02f4-49a1-953b-032c85fdf59e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d62232282dc4455834857d5290eb169'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:22.973235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1399/2804] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:38:22 2022] DELETE /identity/v3/users/2d62232282dc4455834857d5290eb169 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:38:23.147140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4adf3-0a68-472d-bfa1-cee1e2359091 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:23.158805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4adf3-0a68-472d-bfa1-cee1e2359091 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:23.215978 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:23.216614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:23.216933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:23.217245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] PATH_INFO: `/v3/projects/eaa5d21fc40540e4b811713e5efeff46` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:23.218367 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaa5d21fc40540e4b811713e5efeff46)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:23.220888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:23.267219 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaa5d21fc40540e4b811713e5efeff46', 'request_id': 'req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.273065 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaa5d21fc40540e4b811713e5efeff46', 'request_id': 'req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.281916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaa5d21fc40540e4b811713e5efeff46', 'request_id': 'req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.292650 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] The token cache is being invalidate because project eaa5d21fc40540e4b811713e5efeff46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:38:23.292933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.293576 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.295011 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1406/2805] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:38:23 2022] DELETE /identity/v3/projects/eaa5d21fc40540e4b811713e5efeff46 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:38:23.300068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce0cbf-02f4-49a1-953b-032c85fdf59e admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:23.301517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a2ce0cbf-02f4-49a1-953b-032c85fdf59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:23.314835 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce0cbf-02f4-49a1-953b-032c85fdf59e admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:23.371909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2ce66-488c-4ea6-8051-ad7530bc6cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:23.372665 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2ce66-488c-4ea6-8051-ad7530bc6cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:23.372954 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2ce66-488c-4ea6-8051-ad7530bc6cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:23.373231 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2ce66-488c-4ea6-8051-ad7530bc6cd0 admin admin] PATH_INFO: `/v3/users/97719914a8b34a7496e718d40865487b` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:23.374366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2ce66-488c-4ea6-8051-ad7530bc6cd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97719914a8b34a7496e718d40865487b)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:23.376371 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2ce66-488c-4ea6-8051-ad7530bc6cd0 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:23.421540 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-35a2ce66-488c-4ea6-8051-ad7530bc6cd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97719914a8b34a7496e718d40865487b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.434641 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-35a2ce66-488c-4ea6-8051-ad7530bc6cd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97719914a8b34a7496e718d40865487b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.442303 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-35a2ce66-488c-4ea6-8051-ad7530bc6cd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97719914a8b34a7496e718d40865487b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.442920 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-35a2ce66-488c-4ea6-8051-ad7530bc6cd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97719914a8b34a7496e718d40865487b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.450099 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-35a2ce66-488c-4ea6-8051-ad7530bc6cd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97719914a8b34a7496e718d40865487b'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.456760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1400/2806] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:38:23 2022] DELETE /identity/v3/users/97719914a8b34a7496e718d40865487b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:38:23.629460 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:23.630902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:23.648429 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbe875-3edd-45a5-8bf7-b0d96acf8a03 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:23.669437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:23.707616 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c1878-fa7d-472a-bce1-610142314193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:23.708170 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c1878-fa7d-472a-bce1-610142314193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:23.708446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c1878-fa7d-472a-bce1-610142314193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:23.708717 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c1878-fa7d-472a-bce1-610142314193 admin admin] PATH_INFO: `/v3/projects/90b1f65e3cf44a8fae0aed69971e8810` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:23.709816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c1878-fa7d-472a-bce1-610142314193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90b1f65e3cf44a8fae0aed69971e8810)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:23.711722 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c1878-fa7d-472a-bce1-610142314193 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:23.757576 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2c9c1878-fa7d-472a-bce1-610142314193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90b1f65e3cf44a8fae0aed69971e8810', 'request_id': 'req-2c9c1878-fa7d-472a-bce1-610142314193'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.763077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2c9c1878-fa7d-472a-bce1-610142314193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90b1f65e3cf44a8fae0aed69971e8810', 'request_id': 'req-2c9c1878-fa7d-472a-bce1-610142314193'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.770950 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2c9c1878-fa7d-472a-bce1-610142314193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90b1f65e3cf44a8fae0aed69971e8810', 'request_id': 'req-2c9c1878-fa7d-472a-bce1-610142314193'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.782217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2c9c1878-fa7d-472a-bce1-610142314193 admin admin] The token cache is being invalidate because project 90b1f65e3cf44a8fae0aed69971e8810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:38:23.782501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2c9c1878-fa7d-472a-bce1-610142314193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.783126 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2c9c1878-fa7d-472a-bce1-610142314193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:38:23.784602 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1407/2807] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:38:23 2022] DELETE /identity/v3/projects/90b1f65e3cf44a8fae0aed69971e8810 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:38:33.452477 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=62,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:33.761531 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2ce66-488c-4ea6-8051-ad7530bc6cd0 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:33.762932 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-35a2ce66-488c-4ea6-8051-ad7530bc6cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:33.778701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:33.839602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2ce66-488c-4ea6-8051-ad7530bc6cd0 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:33.886017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2bfad-e23d-46d7-b0dd-f54e82120456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:33.886550 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2bfad-e23d-46d7-b0dd-f54e82120456 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:33.886828 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2bfad-e23d-46d7-b0dd-f54e82120456 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:33.887100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2bfad-e23d-46d7-b0dd-f54e82120456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:33.888832 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bbb2bfad-e23d-46d7-b0dd-f54e82120456 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:33.958689 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2bfad-e23d-46d7-b0dd-f54e82120456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:33.960195 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2bfad-e23d-46d7-b0dd-f54e82120456 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:33.971063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1401/2808] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:38:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:38:40.915236 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f77fd-3f9e-42da-b6e6-981057400b3a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:40.915995 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f77fd-3f9e-42da-b6e6-981057400b3a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:40.916215 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f77fd-3f9e-42da-b6e6-981057400b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:40.936424 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e44f77fd-3f9e-42da-b6e6-981057400b3a None None] MFA Rules not processed for user `81501360f082464191731f57d2b6a402`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:40.937627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e44f77fd-3f9e-42da-b6e6-981057400b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:40.991957 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1408/2809] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:38:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:38:43.966163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=83 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:50.819190 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2bfad-e23d-46d7-b0dd-f54e82120456 service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:50.820542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-bbb2bfad-e23d-46d7-b0dd-f54e82120456 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:50.875501 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2bfad-e23d-46d7-b0dd-f54e82120456 service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:50.879025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518af4c-aec1-4630-9e1a-7916d07238f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:50.879649 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518af4c-aec1-4630-9e1a-7916d07238f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:50.879929 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518af4c-aec1-4630-9e1a-7916d07238f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:50.880226 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518af4c-aec1-4630-9e1a-7916d07238f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:50.881897 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2518af4c-aec1-4630-9e1a-7916d07238f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:50.891071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518af4c-aec1-4630-9e1a-7916d07238f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:50.892987 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518af4c-aec1-4630-9e1a-7916d07238f5 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:50.901391 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1402/2810] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:38:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:38:50.985207 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=25 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:54.482443 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f77fd-3f9e-42da-b6e6-981057400b3a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:54.484028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e44f77fd-3f9e-42da-b6e6-981057400b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:54.499584 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f77fd-3f9e-42da-b6e6-981057400b3a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:54.505079 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874f4f7-3b6a-4a33-be13-62d8ba47d9af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:54.505786 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874f4f7-3b6a-4a33-be13-62d8ba47d9af service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:54.506077 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874f4f7-3b6a-4a33-be13-62d8ba47d9af service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:54.506353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874f4f7-3b6a-4a33-be13-62d8ba47d9af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:54.508103 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5874f4f7-3b6a-4a33-be13-62d8ba47d9af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:54.520900 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874f4f7-3b6a-4a33-be13-62d8ba47d9af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:54.522466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874f4f7-3b6a-4a33-be13-62d8ba47d9af service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:54.533788 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1409/2811] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:38:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:39:00.897930 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=26 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:04.528524 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:36.768144 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518af4c-aec1-4630-9e1a-7916d07238f5 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:36.769887 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-2518af4c-aec1-4630-9e1a-7916d07238f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:36.802469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518af4c-aec1-4630-9e1a-7916d07238f5 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:36.811711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749f1af-af3e-443b-a106-228941eae323 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5fa28ffdafd435cb2d93bd453add3cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:36.812403 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749f1af-af3e-443b-a106-228941eae323 service nova] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:36.812688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749f1af-af3e-443b-a106-228941eae323 service nova] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:36.813000 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749f1af-af3e-443b-a106-228941eae323 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:36.815128 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b749f1af-af3e-443b-a106-228941eae323 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:36.828558 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b749f1af-af3e-443b-a106-228941eae323 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:36.830223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b749f1af-af3e-443b-a106-228941eae323 service nova] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:36.848853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1403/2812] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:39:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:39:45.347940 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874f4f7-3b6a-4a33-be13-62d8ba47d9af service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:45.349307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5874f4f7-3b6a-4a33-be13-62d8ba47d9af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:45.420704 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874f4f7-3b6a-4a33-be13-62d8ba47d9af service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:45.425210 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba38f2-9ad2-4a29-b54d-e5a100d91752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:45.425838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba38f2-9ad2-4a29-b54d-e5a100d91752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:45.426118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba38f2-9ad2-4a29-b54d-e5a100d91752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:45.426395 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba38f2-9ad2-4a29-b54d-e5a100d91752 admin admin] PATH_INFO: `/v3/users/81501360f082464191731f57d2b6a402` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:45.427503 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba38f2-9ad2-4a29-b54d-e5a100d91752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81501360f082464191731f57d2b6a402)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:45.429441 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba38f2-9ad2-4a29-b54d-e5a100d91752 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:45.474356 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-19ba38f2-9ad2-4a29-b54d-e5a100d91752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81501360f082464191731f57d2b6a402'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:45.481930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-19ba38f2-9ad2-4a29-b54d-e5a100d91752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81501360f082464191731f57d2b6a402'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:45.493687 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-19ba38f2-9ad2-4a29-b54d-e5a100d91752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81501360f082464191731f57d2b6a402'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:45.501918 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-19ba38f2-9ad2-4a29-b54d-e5a100d91752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81501360f082464191731f57d2b6a402'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:45.502659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-19ba38f2-9ad2-4a29-b54d-e5a100d91752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81501360f082464191731f57d2b6a402'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:45.508602 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1410/2813] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:39:45 2022] DELETE /identity/v3/users/81501360f082464191731f57d2b6a402 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:45.684772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749f1af-af3e-443b-a106-228941eae323 service nova] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:45.697759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749f1af-af3e-443b-a106-228941eae323 service nova] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:45.755593 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69053db7-8601-4532-9c0c-15686736b8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:45.756225 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69053db7-8601-4532-9c0c-15686736b8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:45.756507 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69053db7-8601-4532-9c0c-15686736b8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:45.756785 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69053db7-8601-4532-9c0c-15686736b8dc admin admin] PATH_INFO: `/v3/projects/1649d6cd95d4496baaee3a0c0595a304` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:45.757884 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69053db7-8601-4532-9c0c-15686736b8dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1649d6cd95d4496baaee3a0c0595a304)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:45.759841 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69053db7-8601-4532-9c0c-15686736b8dc admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:45.803925 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-69053db7-8601-4532-9c0c-15686736b8dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1649d6cd95d4496baaee3a0c0595a304', 'request_id': 'req-69053db7-8601-4532-9c0c-15686736b8dc'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:45.809339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-69053db7-8601-4532-9c0c-15686736b8dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1649d6cd95d4496baaee3a0c0595a304', 'request_id': 'req-69053db7-8601-4532-9c0c-15686736b8dc'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:45.816471 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-69053db7-8601-4532-9c0c-15686736b8dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1649d6cd95d4496baaee3a0c0595a304', 'request_id': 'req-69053db7-8601-4532-9c0c-15686736b8dc'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:45.827151 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-69053db7-8601-4532-9c0c-15686736b8dc admin admin] The token cache is being invalidate because project 1649d6cd95d4496baaee3a0c0595a304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:39:45.827442 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-69053db7-8601-4532-9c0c-15686736b8dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:45.828097 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-69053db7-8601-4532-9c0c-15686736b8dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:45.829533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1404/2814] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:39:45 2022] DELETE /identity/v3/projects/1649d6cd95d4496baaee3a0c0595a304 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:45.834347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3baedc-017a-4627-8f77-82695b6ab0f8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:45.834636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3baedc-017a-4627-8f77-82695b6ab0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:45.834917 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3baedc-017a-4627-8f77-82695b6ab0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:45.908964 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9c3baedc-017a-4627-8f77-82695b6ab0f8 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:45.910179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9c3baedc-017a-4627-8f77-82695b6ab0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:45.953338 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:45.969925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1411/2815] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:45.977738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f30b67-e977-4a78-9a21-4fce5b71c49b None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:45.978024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f30b67-e977-4a78-9a21-4fce5b71c49b None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:45.978298 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f30b67-e977-4a78-9a21-4fce5b71c49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:45.998743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-e9f30b67-e977-4a78-9a21-4fce5b71c49b None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:45.999955 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e9f30b67-e977-4a78-9a21-4fce5b71c49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:46.007537 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1405/2816] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:46.011657 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3baedc-017a-4627-8f77-82695b6ab0f8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.022098 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3baedc-017a-4627-8f77-82695b6ab0f8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.027182 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cc468-f4d3-4908-b70f-89a83b7b8f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.027812 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cc468-f4d3-4908-b70f-89a83b7b8f31 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.028092 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cc468-f4d3-4908-b70f-89a83b7b8f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.028364 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cc468-f4d3-4908-b70f-89a83b7b8f31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.028999 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cc468-f4d3-4908-b70f-89a83b7b8f31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:46.029301 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cc468-f4d3-4908-b70f-89a83b7b8f31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.030871 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cc468-f4d3-4908-b70f-89a83b7b8f31 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.042211 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1412/2817] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:46.046323 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f30b67-e977-4a78-9a21-4fce5b71c49b None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.055155 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f30b67-e977-4a78-9a21-4fce5b71c49b None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.060258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaa042-8ce5-4553-a81b-2b3e5cb09a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.060845 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaa042-8ce5-4553-a81b-2b3e5cb09a74 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.061119 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaa042-8ce5-4553-a81b-2b3e5cb09a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.061385 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaa042-8ce5-4553-a81b-2b3e5cb09a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.062020 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eaa042-8ce5-4553-a81b-2b3e5cb09a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.063952 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eaa042-8ce5-4553-a81b-2b3e5cb09a74 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.100362 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1406/2818] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:39:46 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:46.105062 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cc468-f4d3-4908-b70f-89a83b7b8f31 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.114916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cc468-f4d3-4908-b70f-89a83b7b8f31 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.169309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a4770-335e-45ef-9d67-84f21f098e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.169902 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a4770-335e-45ef-9d67-84f21f098e75 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.170179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a4770-335e-45ef-9d67-84f21f098e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.170450 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a4770-335e-45ef-9d67-84f21f098e75 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.171073 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a4770-335e-45ef-9d67-84f21f098e75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.173030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a4770-335e-45ef-9d67-84f21f098e75 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.193854 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1413/2819] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:39:46 2022] POST /identity/v3/users => generated 285 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:46.198244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaa042-8ce5-4553-a81b-2b3e5cb09a74 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.208555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaa042-8ce5-4553-a81b-2b3e5cb09a74 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.213116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8537f-1cdb-48c0-8e74-f164d3f1d174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.213702 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8537f-1cdb-48c0-8e74-f164d3f1d174 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.213975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8537f-1cdb-48c0-8e74-f164d3f1d174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.214242 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8537f-1cdb-48c0-8e74-f164d3f1d174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.214870 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8537f-1cdb-48c0-8e74-f164d3f1d174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.216480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8537f-1cdb-48c0-8e74-f164d3f1d174 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.226032 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1407/2820] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:39:46.230378 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a4770-335e-45ef-9d67-84f21f098e75 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.240010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a4770-335e-45ef-9d67-84f21f098e75 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.245216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6224a-bb23-4149-9c59-d48848cc5296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.245791 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6224a-bb23-4149-9c59-d48848cc5296 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.246066 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6224a-bb23-4149-9c59-d48848cc5296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.246334 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6224a-bb23-4149-9c59-d48848cc5296 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.246968 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6224a-bb23-4149-9c59-d48848cc5296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.248554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6224a-bb23-4149-9c59-d48848cc5296 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.259925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1414/2821] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:39:46.263844 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8537f-1cdb-48c0-8e74-f164d3f1d174 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.274023 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8537f-1cdb-48c0-8e74-f164d3f1d174 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.278743 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13e131-0f77-4221-adbe-953c287012b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.279385 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13e131-0f77-4221-adbe-953c287012b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.279703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13e131-0f77-4221-adbe-953c287012b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.279981 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13e131-0f77-4221-adbe-953c287012b7 admin admin] PATH_INFO: `/v3/projects/a0c848dd07ce416eae283a0d9d87e657/users/083628a62fd34f9790155ca52342dbf5/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.294709 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13e131-0f77-4221-adbe-953c287012b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0c848dd07ce416eae283a0d9d87e657, user_id=083628a62fd34f9790155ca52342dbf5, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.297385 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13e131-0f77-4221-adbe-953c287012b7 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.309756 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1408/2822] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:39:46 2022] PUT /identity/v3/projects/a0c848dd07ce416eae283a0d9d87e657/users/083628a62fd34f9790155ca52342dbf5/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:46.319747 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c58fe-198f-4694-9c63-575823903e47 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.320049 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c58fe-198f-4694-9c63-575823903e47 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.320320 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c58fe-198f-4694-9c63-575823903e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.347263 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-503c58fe-198f-4694-9c63-575823903e47 None None] MFA Rules not processed for user `083628a62fd34f9790155ca52342dbf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:46.348502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-503c58fe-198f-4694-9c63-575823903e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:46.411510 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1415/2823] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:46.416387 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13e131-0f77-4221-adbe-953c287012b7 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.421854 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:46.479360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13e131-0f77-4221-adbe-953c287012b7 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.483991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f9fdb-b308-4fb6-8999-937a32139d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.484587 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f9fdb-b308-4fb6-8999-937a32139d0e admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.484867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f9fdb-b308-4fb6-8999-937a32139d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.485141 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f9fdb-b308-4fb6-8999-937a32139d0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.485764 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f9fdb-b308-4fb6-8999-937a32139d0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.487670 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f9fdb-b308-4fb6-8999-937a32139d0e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.512770 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1409/2824] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:39:46 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:46.516754 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c58fe-198f-4694-9c63-575823903e47 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.527890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c58fe-198f-4694-9c63-575823903e47 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.568711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:46.584118 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab6fa0-b191-4cde-bebe-b30e8ad7cf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.584720 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab6fa0-b191-4cde-bebe-b30e8ad7cf5d admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.585411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab6fa0-b191-4cde-bebe-b30e8ad7cf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.585411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab6fa0-b191-4cde-bebe-b30e8ad7cf5d admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.585969 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab6fa0-b191-4cde-bebe-b30e8ad7cf5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.588638 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab6fa0-b191-4cde-bebe-b30e8ad7cf5d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.611457 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1416/2825] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:39:46 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:46.615462 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f9fdb-b308-4fb6-8999-937a32139d0e admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.625804 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f9fdb-b308-4fb6-8999-937a32139d0e admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.630528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f58d6f-f0c6-4400-a24b-136b7949de5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.631185 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f58d6f-f0c6-4400-a24b-136b7949de5e admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.631463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f58d6f-f0c6-4400-a24b-136b7949de5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.631766 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f58d6f-f0c6-4400-a24b-136b7949de5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.632422 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f58d6f-f0c6-4400-a24b-136b7949de5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.634025 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f58d6f-f0c6-4400-a24b-136b7949de5e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.649527 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1410/2826] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:39:46.653998 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab6fa0-b191-4cde-bebe-b30e8ad7cf5d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.664303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab6fa0-b191-4cde-bebe-b30e8ad7cf5d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.669667 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbcc0b-c7fe-4841-95e7-d03160951de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.670347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbcc0b-c7fe-4841-95e7-d03160951de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.670628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbcc0b-c7fe-4841-95e7-d03160951de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.670906 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbcc0b-c7fe-4841-95e7-d03160951de6 admin admin] PATH_INFO: `/v3/projects/8556ca3b204b44aa81e3962419b7db4a/users/e569f3e053af4848826fe58ef55b2ead/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.686609 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbcc0b-c7fe-4841-95e7-d03160951de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8556ca3b204b44aa81e3962419b7db4a, user_id=e569f3e053af4848826fe58ef55b2ead, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.689274 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbcc0b-c7fe-4841-95e7-d03160951de6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.701496 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1417/2827] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:39:46 2022] PUT /identity/v3/projects/8556ca3b204b44aa81e3962419b7db4a/users/e569f3e053af4848826fe58ef55b2ead/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:46.705631 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f58d6f-f0c6-4400-a24b-136b7949de5e admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.716072 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f58d6f-f0c6-4400-a24b-136b7949de5e admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.770909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe894b62-c57c-4136-8dad-ad289392a8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.771502 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe894b62-c57c-4136-8dad-ad289392a8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.772071 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe894b62-c57c-4136-8dad-ad289392a8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.772339 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe894b62-c57c-4136-8dad-ad289392a8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.772962 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe894b62-c57c-4136-8dad-ad289392a8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.774536 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe894b62-c57c-4136-8dad-ad289392a8d3 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.784368 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1411/2828] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:39:46.788575 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbcc0b-c7fe-4841-95e7-d03160951de6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.799825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbcc0b-c7fe-4841-95e7-d03160951de6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.804535 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650cdce-054f-42bc-87e8-0aac79f3b185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.805182 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650cdce-054f-42bc-87e8-0aac79f3b185 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.805461 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650cdce-054f-42bc-87e8-0aac79f3b185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.805735 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650cdce-054f-42bc-87e8-0aac79f3b185 admin admin] PATH_INFO: `/v3/projects/8556ca3b204b44aa81e3962419b7db4a/users/e569f3e053af4848826fe58ef55b2ead/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.807682 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650cdce-054f-42bc-87e8-0aac79f3b185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8556ca3b204b44aa81e3962419b7db4a, user_id=e569f3e053af4848826fe58ef55b2ead, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.810295 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650cdce-054f-42bc-87e8-0aac79f3b185 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.820666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1418/2829] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:39:46 2022] PUT /identity/v3/projects/8556ca3b204b44aa81e3962419b7db4a/users/e569f3e053af4848826fe58ef55b2ead/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:46.832673 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2f3e5-5ea3-4bf3-b5c5-c27a446aec7d None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.832984 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2f3e5-5ea3-4bf3-b5c5-c27a446aec7d None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.833264 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2f3e5-5ea3-4bf3-b5c5-c27a446aec7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.860821 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-51e2f3e5-5ea3-4bf3-b5c5-c27a446aec7d None None] MFA Rules not processed for user `e569f3e053af4848826fe58ef55b2ead`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:46.862015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-51e2f3e5-5ea3-4bf3-b5c5-c27a446aec7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:46.925546 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:46.929959 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1412/2830] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:46.940347 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650cdce-054f-42bc-87e8-0aac79f3b185 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.942059 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5650cdce-054f-42bc-87e8-0aac79f3b185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:46.998729 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650cdce-054f-42bc-87e8-0aac79f3b185 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:47.002367 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82ed9e-42a5-44d0-a6c3-1e30e271106b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:47.002961 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82ed9e-42a5-44d0-a6c3-1e30e271106b service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:47.003239 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82ed9e-42a5-44d0-a6c3-1e30e271106b service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:47.003511 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82ed9e-42a5-44d0-a6c3-1e30e271106b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:47.012826 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82ed9e-42a5-44d0-a6c3-1e30e271106b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:47.014245 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82ed9e-42a5-44d0-a6c3-1e30e271106b service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:47.023818 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1419/2831] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:39:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:39:48.505985 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2f3e5-5ea3-4bf3-b5c5-c27a446aec7d None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:48.567967 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2f3e5-5ea3-4bf3-b5c5-c27a446aec7d None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:48.572350 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ae146-00ce-468d-a1e7-f7e151aa05cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:48.572969 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ae146-00ce-468d-a1e7-f7e151aa05cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:48.573246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ae146-00ce-468d-a1e7-f7e151aa05cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:48.573524 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ae146-00ce-468d-a1e7-f7e151aa05cf admin admin] PATH_INFO: `/v3/users/083628a62fd34f9790155ca52342dbf5` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:48.574628 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ae146-00ce-468d-a1e7-f7e151aa05cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=083628a62fd34f9790155ca52342dbf5)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:48.576627 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ae146-00ce-468d-a1e7-f7e151aa05cf admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:48.621635 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-154ae146-00ce-468d-a1e7-f7e151aa05cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '083628a62fd34f9790155ca52342dbf5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:48.633251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-154ae146-00ce-468d-a1e7-f7e151aa05cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '083628a62fd34f9790155ca52342dbf5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:48.640191 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-154ae146-00ce-468d-a1e7-f7e151aa05cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '083628a62fd34f9790155ca52342dbf5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:48.640852 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-154ae146-00ce-468d-a1e7-f7e151aa05cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '083628a62fd34f9790155ca52342dbf5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:48.647933 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-154ae146-00ce-468d-a1e7-f7e151aa05cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '083628a62fd34f9790155ca52342dbf5'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:48.653807 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1413/2832] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:39:48 2022] DELETE /identity/v3/users/083628a62fd34f9790155ca52342dbf5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:48.658024 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fd079-6b2d-416e-b656-c8fb60d24507 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:48.658486 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fd079-6b2d-416e-b656-c8fb60d24507 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:48.658608 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fd079-6b2d-416e-b656-c8fb60d24507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:48.677437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b46fd079-6b2d-416e-b656-c8fb60d24507 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:48.678631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b46fd079-6b2d-416e-b656-c8fb60d24507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:48.737470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1420/2833] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:48.747201 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ae146-00ce-468d-a1e7-f7e151aa05cf admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:48.748617 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-154ae146-00ce-468d-a1e7-f7e151aa05cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:48.803815 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ae146-00ce-468d-a1e7-f7e151aa05cf admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:48.807349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e46f78-5ea1-480e-8012-a4602b8dd9b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:48.807963 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e46f78-5ea1-480e-8012-a4602b8dd9b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:48.808240 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e46f78-5ea1-480e-8012-a4602b8dd9b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:48.808508 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e46f78-5ea1-480e-8012-a4602b8dd9b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:48.817506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e46f78-5ea1-480e-8012-a4602b8dd9b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:48.818982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e46f78-5ea1-480e-8012-a4602b8dd9b0 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:48.828253 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1414/2834] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:39:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:39:49.095481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fd079-6b2d-416e-b656-c8fb60d24507 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:49.097084 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b46fd079-6b2d-416e-b656-c8fb60d24507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:49.113862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fd079-6b2d-416e-b656-c8fb60d24507 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:49.122974 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:49.123660 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:49.123938 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:49.124209 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] PATH_INFO: `/v3/projects/a0c848dd07ce416eae283a0d9d87e657` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:49.125410 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0c848dd07ce416eae283a0d9d87e657)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:49.127322 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:49.166083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:49.177926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0c848dd07ce416eae283a0d9d87e657', 'request_id': 'req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.183786 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0c848dd07ce416eae283a0d9d87e657', 'request_id': 'req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.191140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0c848dd07ce416eae283a0d9d87e657', 'request_id': 'req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.202598 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] The token cache is being invalidate because project a0c848dd07ce416eae283a0d9d87e657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:39:49.202888 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.203520 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.205023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1421/2835] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:39:49 2022] DELETE /identity/v3/projects/a0c848dd07ce416eae283a0d9d87e657 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:49.209165 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e46f78-5ea1-480e-8012-a4602b8dd9b0 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:49.219196 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e46f78-5ea1-480e-8012-a4602b8dd9b0 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:49.236702 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:49.276823 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2b5d1-055a-470e-ac71-0c531aa20566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:49.277442 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2b5d1-055a-470e-ac71-0c531aa20566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:49.277718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2b5d1-055a-470e-ac71-0c531aa20566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:49.277988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2b5d1-055a-470e-ac71-0c531aa20566 admin admin] PATH_INFO: `/v3/users/e569f3e053af4848826fe58ef55b2ead` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:49.279079 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2b5d1-055a-470e-ac71-0c531aa20566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e569f3e053af4848826fe58ef55b2ead)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:49.281050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2b5d1-055a-470e-ac71-0c531aa20566 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:49.332026 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9fb2b5d1-055a-470e-ac71-0c531aa20566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e569f3e053af4848826fe58ef55b2ead'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.344361 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9fb2b5d1-055a-470e-ac71-0c531aa20566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e569f3e053af4848826fe58ef55b2ead'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.351360 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9fb2b5d1-055a-470e-ac71-0c531aa20566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e569f3e053af4848826fe58ef55b2ead'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.351999 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9fb2b5d1-055a-470e-ac71-0c531aa20566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e569f3e053af4848826fe58ef55b2ead'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.358774 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9fb2b5d1-055a-470e-ac71-0c531aa20566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e569f3e053af4848826fe58ef55b2ead'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.364533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1415/2836] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:39:49 2022] DELETE /identity/v3/users/e569f3e053af4848826fe58ef55b2ead => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:49.635779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:49.637915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:49.653269 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f4ab2-35d4-4bad-96e8-b74d5fab91a2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:49.710308 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af8662-1f6b-4c28-a445-c894e97462cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:49.710997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8662-1f6b-4c28-a445-c894e97462cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:49.711199 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8662-1f6b-4c28-a445-c894e97462cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:49.711472 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8662-1f6b-4c28-a445-c894e97462cf admin admin] PATH_INFO: `/v3/projects/8556ca3b204b44aa81e3962419b7db4a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:49.712628 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af8662-1f6b-4c28-a445-c894e97462cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8556ca3b204b44aa81e3962419b7db4a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:49.714547 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af8662-1f6b-4c28-a445-c894e97462cf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:49.761277 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-47af8662-1f6b-4c28-a445-c894e97462cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8556ca3b204b44aa81e3962419b7db4a', 'request_id': 'req-47af8662-1f6b-4c28-a445-c894e97462cf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.766646 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-47af8662-1f6b-4c28-a445-c894e97462cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8556ca3b204b44aa81e3962419b7db4a', 'request_id': 'req-47af8662-1f6b-4c28-a445-c894e97462cf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.773760 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-47af8662-1f6b-4c28-a445-c894e97462cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8556ca3b204b44aa81e3962419b7db4a', 'request_id': 'req-47af8662-1f6b-4c28-a445-c894e97462cf'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.784459 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-47af8662-1f6b-4c28-a445-c894e97462cf admin admin] The token cache is being invalidate because project 8556ca3b204b44aa81e3962419b7db4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:39:49.784746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-47af8662-1f6b-4c28-a445-c894e97462cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.785374 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-47af8662-1f6b-4c28-a445-c894e97462cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:49.786834 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1422/2837] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:39:49 2022] DELETE /identity/v3/projects/8556ca3b204b44aa81e3962419b7db4a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:49.791668 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174923b-9f34-4d15-986f-443f4b8b71dc None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:49.792162 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174923b-9f34-4d15-986f-443f4b8b71dc None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:49.792289 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174923b-9f34-4d15-986f-443f4b8b71dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:49.812280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-1174923b-9f34-4d15-986f-443f4b8b71dc None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:49.813482 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-1174923b-9f34-4d15-986f-443f4b8b71dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:49.875200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1416/2838] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:49.883164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4f3e6-0c48-40fa-bb4b-0db2c2d96bd5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:49.883393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4f3e6-0c48-40fa-bb4b-0db2c2d96bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:49.883693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4f3e6-0c48-40fa-bb4b-0db2c2d96bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:49.904707 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-75c4f3e6-0c48-40fa-bb4b-0db2c2d96bd5 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:49.905898 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-75c4f3e6-0c48-40fa-bb4b-0db2c2d96bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:49.914359 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1423/2839] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:49.919116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1174923b-9f34-4d15-986f-443f4b8b71dc None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:49.929107 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1174923b-9f34-4d15-986f-443f4b8b71dc None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:49.934312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fff3f-7ab8-4a39-9765-ad38d6398645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:49.934910 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fff3f-7ab8-4a39-9765-ad38d6398645 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:49.935190 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fff3f-7ab8-4a39-9765-ad38d6398645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:49.935491 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fff3f-7ab8-4a39-9765-ad38d6398645 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:49.936223 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fff3f-7ab8-4a39-9765-ad38d6398645 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:49.936552 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fff3f-7ab8-4a39-9765-ad38d6398645 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:49.938155 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fff3f-7ab8-4a39-9765-ad38d6398645 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:49.949495 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1417/2840] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:49.953914 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4f3e6-0c48-40fa-bb4b-0db2c2d96bd5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:49.963481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4f3e6-0c48-40fa-bb4b-0db2c2d96bd5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:49.968251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbce36-7a70-4203-bec9-c4dfef5e23df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:49.968865 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbce36-7a70-4203-bec9-c4dfef5e23df admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:49.969139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbce36-7a70-4203-bec9-c4dfef5e23df admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:49.969411 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbce36-7a70-4203-bec9-c4dfef5e23df admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:49.970045 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbce36-7a70-4203-bec9-c4dfef5e23df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:49.972044 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbce36-7a70-4203-bec9-c4dfef5e23df admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.000384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1424/2841] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:39:49 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:50.004623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fff3f-7ab8-4a39-9765-ad38d6398645 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.014725 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fff3f-7ab8-4a39-9765-ad38d6398645 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.040181 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:50.073718 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae9d2a-48a3-4510-b1db-90f9be56ddbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.074319 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae9d2a-48a3-4510-b1db-90f9be56ddbe admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.074597 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae9d2a-48a3-4510-b1db-90f9be56ddbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.074868 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae9d2a-48a3-4510-b1db-90f9be56ddbe admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.075499 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae9d2a-48a3-4510-b1db-90f9be56ddbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.077430 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae9d2a-48a3-4510-b1db-90f9be56ddbe admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.098480 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1418/2842] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:39:50 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:50.102683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbce36-7a70-4203-bec9-c4dfef5e23df admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.113349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbce36-7a70-4203-bec9-c4dfef5e23df admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.117976 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafca3be-9ef1-4f40-b425-bfdc45b419e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.118573 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafca3be-9ef1-4f40-b425-bfdc45b419e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.118857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafca3be-9ef1-4f40-b425-bfdc45b419e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.119129 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafca3be-9ef1-4f40-b425-bfdc45b419e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.119807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafca3be-9ef1-4f40-b425-bfdc45b419e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.121381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafca3be-9ef1-4f40-b425-bfdc45b419e5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.132623 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1425/2843] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:39:50.137337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae9d2a-48a3-4510-b1db-90f9be56ddbe admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.147849 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae9d2a-48a3-4510-b1db-90f9be56ddbe admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.153066 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071e43c-4ed7-45b3-9f96-36d45453f5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.153657 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071e43c-4ed7-45b3-9f96-36d45453f5bf admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.153934 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071e43c-4ed7-45b3-9f96-36d45453f5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.154206 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071e43c-4ed7-45b3-9f96-36d45453f5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.154951 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071e43c-4ed7-45b3-9f96-36d45453f5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.156760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071e43c-4ed7-45b3-9f96-36d45453f5bf admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.167630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1419/2844] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:39:50.171820 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafca3be-9ef1-4f40-b425-bfdc45b419e5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.182353 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafca3be-9ef1-4f40-b425-bfdc45b419e5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.187250 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8313cb-50d5-4d17-99c2-32f6fcc35e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.187928 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8313cb-50d5-4d17-99c2-32f6fcc35e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.188273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8313cb-50d5-4d17-99c2-32f6fcc35e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.188507 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8313cb-50d5-4d17-99c2-32f6fcc35e3e admin admin] PATH_INFO: `/v3/projects/44caec37c2c84e578c16fe09a6671651/users/339575f4630f4e71bb936b5ef8cbf800/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.200135 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:50.204807 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8313cb-50d5-4d17-99c2-32f6fcc35e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=44caec37c2c84e578c16fe09a6671651, user_id=339575f4630f4e71bb936b5ef8cbf800, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.208440 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8313cb-50d5-4d17-99c2-32f6fcc35e3e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.221728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1426/2845] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:39:50 2022] PUT /identity/v3/projects/44caec37c2c84e578c16fe09a6671651/users/339575f4630f4e71bb936b5ef8cbf800/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:50.233516 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af77aa-afbd-4b24-9666-8ec6e51f7f95 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.233805 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af77aa-afbd-4b24-9666-8ec6e51f7f95 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.234076 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af77aa-afbd-4b24-9666-8ec6e51f7f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.262489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-12af77aa-afbd-4b24-9666-8ec6e51f7f95 None None] MFA Rules not processed for user `339575f4630f4e71bb936b5ef8cbf800`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:50.263731 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-12af77aa-afbd-4b24-9666-8ec6e51f7f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:50.328676 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1420/2846] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:50.333140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8313cb-50d5-4d17-99c2-32f6fcc35e3e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.395097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8313cb-50d5-4d17-99c2-32f6fcc35e3e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.399923 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66bf86-459f-40c4-b1f8-b9103762d699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.400532 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66bf86-459f-40c4-b1f8-b9103762d699 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.400813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66bf86-459f-40c4-b1f8-b9103762d699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.401089 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66bf86-459f-40c4-b1f8-b9103762d699 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.401813 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66bf86-459f-40c4-b1f8-b9103762d699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.404519 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66bf86-459f-40c4-b1f8-b9103762d699 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.430711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1427/2847] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:39:50 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:50.435309 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af77aa-afbd-4b24-9666-8ec6e51f7f95 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.446063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af77aa-afbd-4b24-9666-8ec6e51f7f95 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.502303 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08867da4-3f9f-44b2-b741-d2bdb7506a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.502938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867da4-3f9f-44b2-b741-d2bdb7506a29 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.503217 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867da4-3f9f-44b2-b741-d2bdb7506a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.503489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867da4-3f9f-44b2-b741-d2bdb7506a29 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.504152 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08867da4-3f9f-44b2-b741-d2bdb7506a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.506066 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08867da4-3f9f-44b2-b741-d2bdb7506a29 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.528103 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1421/2848] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:39:50 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:50.532191 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66bf86-459f-40c4-b1f8-b9103762d699 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.542561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66bf86-459f-40c4-b1f8-b9103762d699 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.547232 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590ccc9-d7a8-43a4-8525-389d6f6f8ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.547863 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590ccc9-d7a8-43a4-8525-389d6f6f8ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.548136 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590ccc9-d7a8-43a4-8525-389d6f6f8ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.548406 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590ccc9-d7a8-43a4-8525-389d6f6f8ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.549032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590ccc9-d7a8-43a4-8525-389d6f6f8ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.550604 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590ccc9-d7a8-43a4-8525-389d6f6f8ce5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.561328 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1428/2849] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:39:50.565244 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08867da4-3f9f-44b2-b741-d2bdb7506a29 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.575680 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08867da4-3f9f-44b2-b741-d2bdb7506a29 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.580269 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4592dd-44d0-4c8f-917d-85a1dd73cc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.580853 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4592dd-44d0-4c8f-917d-85a1dd73cc16 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.581127 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4592dd-44d0-4c8f-917d-85a1dd73cc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.581394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4592dd-44d0-4c8f-917d-85a1dd73cc16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.582080 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4592dd-44d0-4c8f-917d-85a1dd73cc16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.583671 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4592dd-44d0-4c8f-917d-85a1dd73cc16 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.593485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1422/2850] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:39:50.597512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590ccc9-d7a8-43a4-8525-389d6f6f8ce5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.608140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590ccc9-d7a8-43a4-8525-389d6f6f8ce5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.612693 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92844be7-fdc4-46b9-827f-96fdfea1e97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.613332 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92844be7-fdc4-46b9-827f-96fdfea1e97f admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.613612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92844be7-fdc4-46b9-827f-96fdfea1e97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.613884 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92844be7-fdc4-46b9-827f-96fdfea1e97f admin admin] PATH_INFO: `/v3/projects/015f1d4923194985a7efdd6a54667fb2/users/fcb8082449f441dba2715d3161677110/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.628684 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92844be7-fdc4-46b9-827f-96fdfea1e97f admin admin] RBAC: Authorizing `identity:create_grant(project_id=015f1d4923194985a7efdd6a54667fb2, user_id=fcb8082449f441dba2715d3161677110, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.631294 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92844be7-fdc4-46b9-827f-96fdfea1e97f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.645234 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1429/2851] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:39:50 2022] PUT /identity/v3/projects/015f1d4923194985a7efdd6a54667fb2/users/fcb8082449f441dba2715d3161677110/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:50.655450 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876fee6-d9fa-44ea-8976-210dfbddb16a None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.655760 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876fee6-d9fa-44ea-8976-210dfbddb16a None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.656031 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876fee6-d9fa-44ea-8976-210dfbddb16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.683881 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-8876fee6-d9fa-44ea-8976-210dfbddb16a None None] MFA Rules not processed for user `fcb8082449f441dba2715d3161677110`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:50.685078 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-8876fee6-d9fa-44ea-8976-210dfbddb16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:50.688941 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:50.750409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1423/2852] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:50.754939 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92844be7-fdc4-46b9-827f-96fdfea1e97f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.814974 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92844be7-fdc4-46b9-827f-96fdfea1e97f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.819613 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1be200-12fe-4837-b4fa-f96d96adbc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.820212 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1be200-12fe-4837-b4fa-f96d96adbc23 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.820492 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1be200-12fe-4837-b4fa-f96d96adbc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.820766 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1be200-12fe-4837-b4fa-f96d96adbc23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.821384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1be200-12fe-4837-b4fa-f96d96adbc23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.823267 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1be200-12fe-4837-b4fa-f96d96adbc23 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.848515 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1430/2853] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:39:50 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:50.852700 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876fee6-d9fa-44ea-8976-210dfbddb16a None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.863588 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876fee6-d9fa-44ea-8976-210dfbddb16a None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.919341 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc632d7e-2e3e-453c-835a-15f18702708f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.920002 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc632d7e-2e3e-453c-835a-15f18702708f admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.920284 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc632d7e-2e3e-453c-835a-15f18702708f admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.920555 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc632d7e-2e3e-453c-835a-15f18702708f admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.921183 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc632d7e-2e3e-453c-835a-15f18702708f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.923595 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc632d7e-2e3e-453c-835a-15f18702708f admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.944517 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1424/2854] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:39:50 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:39:50.949010 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1be200-12fe-4837-b4fa-f96d96adbc23 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.959228 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1be200-12fe-4837-b4fa-f96d96adbc23 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.964323 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855da800-1c30-422b-9f0a-56a8337d3421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.964910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855da800-1c30-422b-9f0a-56a8337d3421 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.965192 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855da800-1c30-422b-9f0a-56a8337d3421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.965466 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855da800-1c30-422b-9f0a-56a8337d3421 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.966153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855da800-1c30-422b-9f0a-56a8337d3421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.967746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855da800-1c30-422b-9f0a-56a8337d3421 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.977965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1431/2855] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:39:50.982411 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc632d7e-2e3e-453c-835a-15f18702708f admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.992188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc632d7e-2e3e-453c-835a-15f18702708f admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.996730 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf31408-2fec-4665-b46a-af09071b04a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.997357 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf31408-2fec-4665-b46a-af09071b04a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.997634 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf31408-2fec-4665-b46a-af09071b04a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.997904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf31408-2fec-4665-b46a-af09071b04a1 admin admin] PATH_INFO: `/v3/projects/2b31e5ddd50d49c0ab0d084e39c777d5/users/e07947a898df4a0eb4fb822a4d2af5ff/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:51.012353 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf31408-2fec-4665-b46a-af09071b04a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b31e5ddd50d49c0ab0d084e39c777d5, user_id=e07947a898df4a0eb4fb822a4d2af5ff, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:51.015049 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf31408-2fec-4665-b46a-af09071b04a1 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:51.028162 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1425/2856] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:39:50 2022] PUT /identity/v3/projects/2b31e5ddd50d49c0ab0d084e39c777d5/users/e07947a898df4a0eb4fb822a4d2af5ff/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:51.031913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855da800-1c30-422b-9f0a-56a8337d3421 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:51.042206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855da800-1c30-422b-9f0a-56a8337d3421 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:51.051288 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:51.099105 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f1df9-441f-4a17-8f79-e6c7fbb0f292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:51.099749 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f1df9-441f-4a17-8f79-e6c7fbb0f292 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:51.100032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f1df9-441f-4a17-8f79-e6c7fbb0f292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:51.100307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f1df9-441f-4a17-8f79-e6c7fbb0f292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:51.100935 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f1df9-441f-4a17-8f79-e6c7fbb0f292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:51.102526 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f1df9-441f-4a17-8f79-e6c7fbb0f292 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:51.112612 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1432/2857] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:39:51.116876 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf31408-2fec-4665-b46a-af09071b04a1 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:51.126979 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf31408-2fec-4665-b46a-af09071b04a1 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:51.131583 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1774ab0-bd8e-4bd1-8724-e6d222d283b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:51.132249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1774ab0-bd8e-4bd1-8724-e6d222d283b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:51.132530 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1774ab0-bd8e-4bd1-8724-e6d222d283b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:51.132802 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1774ab0-bd8e-4bd1-8724-e6d222d283b0 admin admin] PATH_INFO: `/v3/projects/2b31e5ddd50d49c0ab0d084e39c777d5/users/e07947a898df4a0eb4fb822a4d2af5ff/roles/23538dc4c770454e80dd11ed8e498539` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:51.134688 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1774ab0-bd8e-4bd1-8724-e6d222d283b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b31e5ddd50d49c0ab0d084e39c777d5, user_id=e07947a898df4a0eb4fb822a4d2af5ff, role_id=23538dc4c770454e80dd11ed8e498539)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:51.137337 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1774ab0-bd8e-4bd1-8724-e6d222d283b0 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:51.147819 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1426/2858] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:39:51 2022] PUT /identity/v3/projects/2b31e5ddd50d49c0ab0d084e39c777d5/users/e07947a898df4a0eb4fb822a4d2af5ff/roles/23538dc4c770454e80dd11ed8e498539 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:51.158171 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37520bf-de90-4105-a419-589ded8be8ea None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:51.158462 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37520bf-de90-4105-a419-589ded8be8ea None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:51.158741 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37520bf-de90-4105-a419-589ded8be8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:51.186832 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a37520bf-de90-4105-a419-589ded8be8ea None None] MFA Rules not processed for user `e07947a898df4a0eb4fb822a4d2af5ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:51.188013 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a37520bf-de90-4105-a419-589ded8be8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:51.253521 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1433/2859] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:51.264585 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1774ab0-bd8e-4bd1-8724-e6d222d283b0 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:51.266303 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-e1774ab0-bd8e-4bd1-8724-e6d222d283b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:51.323899 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1774ab0-bd8e-4bd1-8724-e6d222d283b0 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:51.327650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a799e0-d47a-4af5-9715-f40654898c24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:51.328241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a799e0-d47a-4af5-9715-f40654898c24 service cinder] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:51.328522 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a799e0-d47a-4af5-9715-f40654898c24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:51.328794 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a799e0-d47a-4af5-9715-f40654898c24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:51.338578 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a799e0-d47a-4af5-9715-f40654898c24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:51.340519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a799e0-d47a-4af5-9715-f40654898c24 service cinder] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:51.348281 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:51.353661 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1427/2860] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:39:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:39:51.398020 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37520bf-de90-4105-a419-589ded8be8ea None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:51.399400 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a37520bf-de90-4105-a419-589ded8be8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:51.411897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37520bf-de90-4105-a419-589ded8be8ea None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:51.415446 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d4934-b72c-408e-b354-4c0ef8386572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:51.416092 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d4934-b72c-408e-b354-4c0ef8386572 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:51.416372 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d4934-b72c-408e-b354-4c0ef8386572 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:51.416642 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d4934-b72c-408e-b354-4c0ef8386572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:51.471512 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d4934-b72c-408e-b354-4c0ef8386572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:51.472961 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d4934-b72c-408e-b354-4c0ef8386572 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:51.481675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1434/2861] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:39:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:39:54.354384 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a799e0-d47a-4af5-9715-f40654898c24 service cinder] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:54.414613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a799e0-d47a-4af5-9715-f40654898c24 service cinder] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:54.419078 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cc6e3-3135-420e-89c2-8e5f91d4bd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:54.419758 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cc6e3-3135-420e-89c2-8e5f91d4bd2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:54.420036 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cc6e3-3135-420e-89c2-8e5f91d4bd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:54.420306 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cc6e3-3135-420e-89c2-8e5f91d4bd2d admin admin] PATH_INFO: `/v3/users/339575f4630f4e71bb936b5ef8cbf800` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:54.421407 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cc6e3-3135-420e-89c2-8e5f91d4bd2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=339575f4630f4e71bb936b5ef8cbf800)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:54.423308 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cc6e3-3135-420e-89c2-8e5f91d4bd2d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:54.467859 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-a62cc6e3-3135-420e-89c2-8e5f91d4bd2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '339575f4630f4e71bb936b5ef8cbf800'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:54.479386 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-a62cc6e3-3135-420e-89c2-8e5f91d4bd2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '339575f4630f4e71bb936b5ef8cbf800'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:54.487408 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-a62cc6e3-3135-420e-89c2-8e5f91d4bd2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '339575f4630f4e71bb936b5ef8cbf800'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:54.487991 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-a62cc6e3-3135-420e-89c2-8e5f91d4bd2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '339575f4630f4e71bb936b5ef8cbf800'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:54.494759 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-a62cc6e3-3135-420e-89c2-8e5f91d4bd2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '339575f4630f4e71bb936b5ef8cbf800'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:54.500451 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1428/2862] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:39:54 2022] DELETE /identity/v3/users/339575f4630f4e71bb936b5ef8cbf800 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:54.504899 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0528f-84e5-41fb-8a64-be0c603b1951 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:54.505387 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0528f-84e5-41fb-8a64-be0c603b1951 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:54.505491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0528f-84e5-41fb-8a64-be0c603b1951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:54.526241 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a0e0528f-84e5-41fb-8a64-be0c603b1951 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:54.527361 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a0e0528f-84e5-41fb-8a64-be0c603b1951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:54.554575 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:54.584809 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1435/2863] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:54.594990 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cc6e3-3135-420e-89c2-8e5f91d4bd2d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:54.596602 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-a62cc6e3-3135-420e-89c2-8e5f91d4bd2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:54.652712 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cc6e3-3135-420e-89c2-8e5f91d4bd2d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:54.656330 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b298a-9dc1-4503-af8a-ed0257979ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:54.656947 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b298a-9dc1-4503-af8a-ed0257979ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:54.657222 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b298a-9dc1-4503-af8a-ed0257979ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:54.657492 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b298a-9dc1-4503-af8a-ed0257979ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:54.666427 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b298a-9dc1-4503-af8a-ed0257979ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:54.667891 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b298a-9dc1-4503-af8a-ed0257979ea3 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:54.677086 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1429/2864] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:39:54.949007 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0528f-84e5-41fb-8a64-be0c603b1951 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:54.950510 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a0e0528f-84e5-41fb-8a64-be0c603b1951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:54.962870 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0528f-84e5-41fb-8a64-be0c603b1951 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:54.967701 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:54.968421 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:54.968698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:54.968970 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] PATH_INFO: `/v3/projects/44caec37c2c84e578c16fe09a6671651` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:54.970083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44caec37c2c84e578c16fe09a6671651)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:54.972090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:55.018084 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44caec37c2c84e578c16fe09a6671651', 'request_id': 'req-517761d8-a475-4290-8886-029eeabbc8b9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.023814 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44caec37c2c84e578c16fe09a6671651', 'request_id': 'req-517761d8-a475-4290-8886-029eeabbc8b9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.030924 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44caec37c2c84e578c16fe09a6671651', 'request_id': 'req-517761d8-a475-4290-8886-029eeabbc8b9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.041541 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] The token cache is being invalidate because project 44caec37c2c84e578c16fe09a6671651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:39:55.041831 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.042453 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.043948 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1436/2865] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:39:54 2022] DELETE /identity/v3/projects/44caec37c2c84e578c16fe09a6671651 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:55.047983 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b298a-9dc1-4503-af8a-ed0257979ea3 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:55.058129 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b298a-9dc1-4503-af8a-ed0257979ea3 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:55.076428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:55.115118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fbfdf-e010-43fb-a451-bcdca4fd495e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:55.115757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fbfdf-e010-43fb-a451-bcdca4fd495e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:55.116037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fbfdf-e010-43fb-a451-bcdca4fd495e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:55.116310 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fbfdf-e010-43fb-a451-bcdca4fd495e admin admin] PATH_INFO: `/v3/users/fcb8082449f441dba2715d3161677110` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:55.117404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fbfdf-e010-43fb-a451-bcdca4fd495e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcb8082449f441dba2715d3161677110)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:55.119988 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fbfdf-e010-43fb-a451-bcdca4fd495e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:55.164358 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-242fbfdf-e010-43fb-a451-bcdca4fd495e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcb8082449f441dba2715d3161677110'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.176609 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-242fbfdf-e010-43fb-a451-bcdca4fd495e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcb8082449f441dba2715d3161677110'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.184623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-242fbfdf-e010-43fb-a451-bcdca4fd495e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcb8082449f441dba2715d3161677110'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.185374 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-242fbfdf-e010-43fb-a451-bcdca4fd495e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcb8082449f441dba2715d3161677110'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.192449 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-242fbfdf-e010-43fb-a451-bcdca4fd495e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcb8082449f441dba2715d3161677110'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.198311 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1430/2866] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:39:55 2022] DELETE /identity/v3/users/fcb8082449f441dba2715d3161677110 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:39:55.463916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:55.465344 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:55.478491 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517761d8-a475-4290-8886-029eeabbc8b9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:55.536772 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:55.537457 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:55.537666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:55.537938 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] PATH_INFO: `/v3/projects/015f1d4923194985a7efdd6a54667fb2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:55.539038 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] RBAC: Authorizing `identity:delete_project(project_id=015f1d4923194985a7efdd6a54667fb2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:55.540994 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:55.586587 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '015f1d4923194985a7efdd6a54667fb2', 'request_id': 'req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.591962 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '015f1d4923194985a7efdd6a54667fb2', 'request_id': 'req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.593579 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:55.599776 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '015f1d4923194985a7efdd6a54667fb2', 'request_id': 'req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.611116 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] The token cache is being invalidate because project 015f1d4923194985a7efdd6a54667fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:39:55.611405 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.612060 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.613509 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1437/2867] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:39:55 2022] DELETE /identity/v3/projects/015f1d4923194985a7efdd6a54667fb2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:55.617364 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fbfdf-e010-43fb-a451-bcdca4fd495e admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:55.618754 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-242fbfdf-e010-43fb-a451-bcdca4fd495e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:55.631613 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fbfdf-e010-43fb-a451-bcdca4fd495e admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:55.686287 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f0484-f686-4b6b-a847-73aba6cb0f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:55.686909 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f0484-f686-4b6b-a847-73aba6cb0f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:55.687186 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f0484-f686-4b6b-a847-73aba6cb0f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:55.687458 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f0484-f686-4b6b-a847-73aba6cb0f96 admin admin] PATH_INFO: `/v3/users/e07947a898df4a0eb4fb822a4d2af5ff` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:55.688592 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f0484-f686-4b6b-a847-73aba6cb0f96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e07947a898df4a0eb4fb822a4d2af5ff)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:55.690502 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f0484-f686-4b6b-a847-73aba6cb0f96 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:55.732957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:55.736123 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-353f0484-f686-4b6b-a847-73aba6cb0f96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e07947a898df4a0eb4fb822a4d2af5ff'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.749176 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-353f0484-f686-4b6b-a847-73aba6cb0f96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e07947a898df4a0eb4fb822a4d2af5ff'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.756298 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-353f0484-f686-4b6b-a847-73aba6cb0f96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e07947a898df4a0eb4fb822a4d2af5ff'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.756907 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-353f0484-f686-4b6b-a847-73aba6cb0f96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e07947a898df4a0eb4fb822a4d2af5ff'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.764297 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-353f0484-f686-4b6b-a847-73aba6cb0f96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e07947a898df4a0eb4fb822a4d2af5ff'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:55.770009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1431/2868] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:39:55 2022] DELETE /identity/v3/users/e07947a898df4a0eb4fb822a4d2af5ff => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:56.032759 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.034217 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:56.046921 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbf4df-2d97-4ac2-8b71-2225eaa1564a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.105282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6af29f-4e22-4993-841b-08f0a99d2cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.105916 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6af29f-4e22-4993-841b-08f0a99d2cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.106203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6af29f-4e22-4993-841b-08f0a99d2cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.106481 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6af29f-4e22-4993-841b-08f0a99d2cf5 admin admin] PATH_INFO: `/v3/projects/2b31e5ddd50d49c0ab0d084e39c777d5` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.107627 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6af29f-4e22-4993-841b-08f0a99d2cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b31e5ddd50d49c0ab0d084e39c777d5)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:56.109523 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6af29f-4e22-4993-841b-08f0a99d2cf5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:56.154910 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6f6af29f-4e22-4993-841b-08f0a99d2cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b31e5ddd50d49c0ab0d084e39c777d5', 'request_id': 'req-6f6af29f-4e22-4993-841b-08f0a99d2cf5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:56.160297 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6f6af29f-4e22-4993-841b-08f0a99d2cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b31e5ddd50d49c0ab0d084e39c777d5', 'request_id': 'req-6f6af29f-4e22-4993-841b-08f0a99d2cf5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:56.167302 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6f6af29f-4e22-4993-841b-08f0a99d2cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b31e5ddd50d49c0ab0d084e39c777d5', 'request_id': 'req-6f6af29f-4e22-4993-841b-08f0a99d2cf5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:56.177907 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6f6af29f-4e22-4993-841b-08f0a99d2cf5 admin admin] The token cache is being invalidate because project 2b31e5ddd50d49c0ab0d084e39c777d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:39:56.178193 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6f6af29f-4e22-4993-841b-08f0a99d2cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:56.178805 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6f6af29f-4e22-4993-841b-08f0a99d2cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:56.180282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1438/2869] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:39:56 2022] DELETE /identity/v3/projects/2b31e5ddd50d49c0ab0d084e39c777d5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:39:56.186695 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3dde8-b248-490a-a9bb-0164a94ee83a None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.187100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3dde8-b248-490a-a9bb-0164a94ee83a None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.187484 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3dde8-b248-490a-a9bb-0164a94ee83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.207312 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-dba3dde8-b248-490a-a9bb-0164a94ee83a None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:56.208553 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-dba3dde8-b248-490a-a9bb-0164a94ee83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:56.266215 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1432/2870] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:56.328713 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518336cf-578d-4dcc-a92e-66d4f891d5ec None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.328997 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518336cf-578d-4dcc-a92e-66d4f891d5ec None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.329275 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518336cf-578d-4dcc-a92e-66d4f891d5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.348540 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-518336cf-578d-4dcc-a92e-66d4f891d5ec None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:56.349732 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-518336cf-578d-4dcc-a92e-66d4f891d5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:56.357470 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1439/2871] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:56.361523 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3dde8-b248-490a-a9bb-0164a94ee83a None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.371755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3dde8-b248-490a-a9bb-0164a94ee83a None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.376326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecc6ab-b665-466c-9977-980c196da62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.376921 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecc6ab-b665-466c-9977-980c196da62c admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.377198 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecc6ab-b665-466c-9977-980c196da62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.377469 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecc6ab-b665-466c-9977-980c196da62c admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.378134 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecc6ab-b665-466c-9977-980c196da62c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:56.378432 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecc6ab-b665-466c-9977-980c196da62c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:56.380083 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecc6ab-b665-466c-9977-980c196da62c admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:56.392207 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1433/2872] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:56.396414 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518336cf-578d-4dcc-a92e-66d4f891d5ec None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.405838 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518336cf-578d-4dcc-a92e-66d4f891d5ec None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.410483 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b474be5-ad3a-43fb-8308-a1198b077b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.411074 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b474be5-ad3a-43fb-8308-a1198b077b33 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.411352 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b474be5-ad3a-43fb-8308-a1198b077b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.411658 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b474be5-ad3a-43fb-8308-a1198b077b33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.412294 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b474be5-ad3a-43fb-8308-a1198b077b33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:56.414179 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b474be5-ad3a-43fb-8308-a1198b077b33 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:56.440802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1440/2873] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:39:56 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:56.444848 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecc6ab-b665-466c-9977-980c196da62c admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.455511 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecc6ab-b665-466c-9977-980c196da62c admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.510326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e1b6e-bbdf-45cc-abb7-dbc75fd36b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.510913 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1b6e-bbdf-45cc-abb7-dbc75fd36b33 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.511187 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1b6e-bbdf-45cc-abb7-dbc75fd36b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.511452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1b6e-bbdf-45cc-abb7-dbc75fd36b33 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.512097 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e1b6e-bbdf-45cc-abb7-dbc75fd36b33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:56.514003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e1b6e-bbdf-45cc-abb7-dbc75fd36b33 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:56.534437 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1434/2874] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:39:56 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:56.538463 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b474be5-ad3a-43fb-8308-a1198b077b33 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.548685 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b474be5-ad3a-43fb-8308-a1198b077b33 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.553273 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ef34d-1f48-4e9d-82b9-98934f0afd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.553857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ef34d-1f48-4e9d-82b9-98934f0afd9b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.554133 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ef34d-1f48-4e9d-82b9-98934f0afd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.554433 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ef34d-1f48-4e9d-82b9-98934f0afd9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.555071 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ef34d-1f48-4e9d-82b9-98934f0afd9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:56.557197 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ef34d-1f48-4e9d-82b9-98934f0afd9b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:56.567032 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1441/2875] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:39:56.570867 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e1b6e-bbdf-45cc-abb7-dbc75fd36b33 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.591496 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e1b6e-bbdf-45cc-abb7-dbc75fd36b33 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.596468 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c8809-978e-4213-a15f-3dc99191eaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.597102 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c8809-978e-4213-a15f-3dc99191eaea admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.597358 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c8809-978e-4213-a15f-3dc99191eaea admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.597634 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c8809-978e-4213-a15f-3dc99191eaea admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.598282 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c8809-978e-4213-a15f-3dc99191eaea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:56.602452 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c8809-978e-4213-a15f-3dc99191eaea admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:56.616808 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1435/2876] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:39:56.621778 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ef34d-1f48-4e9d-82b9-98934f0afd9b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.631653 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ef34d-1f48-4e9d-82b9-98934f0afd9b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.636222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e57f23-01ef-45a7-b09a-157e25af8eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.636857 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e57f23-01ef-45a7-b09a-157e25af8eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.637134 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e57f23-01ef-45a7-b09a-157e25af8eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.637405 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e57f23-01ef-45a7-b09a-157e25af8eb9 admin admin] PATH_INFO: `/v3/projects/43728569b115408685346657eec046cf/users/438cf13078d94483ab6eff84a2e619fb/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.651691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e57f23-01ef-45a7-b09a-157e25af8eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43728569b115408685346657eec046cf, user_id=438cf13078d94483ab6eff84a2e619fb, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:56.654307 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e57f23-01ef-45a7-b09a-157e25af8eb9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:56.668182 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1442/2877] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:39:56 2022] PUT /identity/v3/projects/43728569b115408685346657eec046cf/users/438cf13078d94483ab6eff84a2e619fb/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:39:56.678665 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3550-115e-4cbe-a290-298137859263 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.679140 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3550-115e-4cbe-a290-298137859263 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.679246 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3550-115e-4cbe-a290-298137859263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.706568 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-04dc3550-115e-4cbe-a290-298137859263 None None] MFA Rules not processed for user `438cf13078d94483ab6eff84a2e619fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:56.707801 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-04dc3550-115e-4cbe-a290-298137859263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:56.719924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:56.771682 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1436/2878] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:56.783457 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e57f23-01ef-45a7-b09a-157e25af8eb9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.785176 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d0e57f23-01ef-45a7-b09a-157e25af8eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:56.801147 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:56.845247 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e57f23-01ef-45a7-b09a-157e25af8eb9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.854028 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e21e09-69f3-4c11-9f72-3d225394bfb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.854655 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e21e09-69f3-4c11-9f72-3d225394bfb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.855206 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e21e09-69f3-4c11-9f72-3d225394bfb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.855487 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e21e09-69f3-4c11-9f72-3d225394bfb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.864516 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e21e09-69f3-4c11-9f72-3d225394bfb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:56.865942 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e21e09-69f3-4c11-9f72-3d225394bfb2 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:56.879787 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1443/2879] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:39:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:39:59.432567 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc3550-115e-4cbe-a290-298137859263 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:59.492711 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc3550-115e-4cbe-a290-298137859263 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:59.497177 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831449e6-c69a-4223-bf0b-82e00ab7ae0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:59.497798 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831449e6-c69a-4223-bf0b-82e00ab7ae0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:59.498078 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831449e6-c69a-4223-bf0b-82e00ab7ae0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:59.498351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831449e6-c69a-4223-bf0b-82e00ab7ae0d admin admin] PATH_INFO: `/v3/users/438cf13078d94483ab6eff84a2e619fb` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:59.499441 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831449e6-c69a-4223-bf0b-82e00ab7ae0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=438cf13078d94483ab6eff84a2e619fb)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:59.501369 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831449e6-c69a-4223-bf0b-82e00ab7ae0d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:59.546763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-831449e6-c69a-4223-bf0b-82e00ab7ae0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438cf13078d94483ab6eff84a2e619fb'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:59.558258 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-831449e6-c69a-4223-bf0b-82e00ab7ae0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438cf13078d94483ab6eff84a2e619fb'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:59.565788 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-831449e6-c69a-4223-bf0b-82e00ab7ae0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438cf13078d94483ab6eff84a2e619fb'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:59.566542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-831449e6-c69a-4223-bf0b-82e00ab7ae0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438cf13078d94483ab6eff84a2e619fb'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:59.573957 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-831449e6-c69a-4223-bf0b-82e00ab7ae0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438cf13078d94483ab6eff84a2e619fb'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:39:59.579796 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1437/2880] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:39:59 2022] DELETE /identity/v3/users/438cf13078d94483ab6eff84a2e619fb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:59.584350 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa91905-b4a5-4b67-bca7-d5efd71f1bcd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:59.584827 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa91905-b4a5-4b67-bca7-d5efd71f1bcd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:59.584932 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa91905-b4a5-4b67-bca7-d5efd71f1bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:59.604380 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4aa91905-b4a5-4b67-bca7-d5efd71f1bcd None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:59.605578 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4aa91905-b4a5-4b67-bca7-d5efd71f1bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:59.663431 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1444/2881] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:39:59.673436 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831449e6-c69a-4223-bf0b-82e00ab7ae0d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:59.674825 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-831449e6-c69a-4223-bf0b-82e00ab7ae0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:59.714303 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:59.734394 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831449e6-c69a-4223-bf0b-82e00ab7ae0d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:59.738118 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da66ee-d33d-4ec4-b894-f5e9b9733abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:59.738728 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da66ee-d33d-4ec4-b894-f5e9b9733abc service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:59.739012 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da66ee-d33d-4ec4-b894-f5e9b9733abc service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:59.739875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da66ee-d33d-4ec4-b894-f5e9b9733abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:59.750021 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da66ee-d33d-4ec4-b894-f5e9b9733abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:59.751463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da66ee-d33d-4ec4-b894-f5e9b9733abc service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:59.761850 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1438/2882] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:40:00.024931 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa91905-b4a5-4b67-bca7-d5efd71f1bcd None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:00.026484 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4aa91905-b4a5-4b67-bca7-d5efd71f1bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:00.039502 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa91905-b4a5-4b67-bca7-d5efd71f1bcd None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:00.044240 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586949e-7ec9-4b60-80a7-0a34125a7fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:00.044864 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586949e-7ec9-4b60-80a7-0a34125a7fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.045140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586949e-7ec9-4b60-80a7-0a34125a7fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.045409 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586949e-7ec9-4b60-80a7-0a34125a7fb9 admin admin] PATH_INFO: `/v3/projects/43728569b115408685346657eec046cf` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.046647 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586949e-7ec9-4b60-80a7-0a34125a7fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43728569b115408685346657eec046cf)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:00.048595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586949e-7ec9-4b60-80a7-0a34125a7fb9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:00.096105 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1586949e-7ec9-4b60-80a7-0a34125a7fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43728569b115408685346657eec046cf', 'request_id': 'req-1586949e-7ec9-4b60-80a7-0a34125a7fb9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:00.101633 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1586949e-7ec9-4b60-80a7-0a34125a7fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43728569b115408685346657eec046cf', 'request_id': 'req-1586949e-7ec9-4b60-80a7-0a34125a7fb9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:00.108453 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1586949e-7ec9-4b60-80a7-0a34125a7fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43728569b115408685346657eec046cf', 'request_id': 'req-1586949e-7ec9-4b60-80a7-0a34125a7fb9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:00.120019 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1586949e-7ec9-4b60-80a7-0a34125a7fb9 admin admin] The token cache is being invalidate because project 43728569b115408685346657eec046cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:40:00.120303 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1586949e-7ec9-4b60-80a7-0a34125a7fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:00.120925 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1586949e-7ec9-4b60-80a7-0a34125a7fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:00.122384 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1445/2883] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:40:00 2022] DELETE /identity/v3/projects/43728569b115408685346657eec046cf => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:40:00.127163 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec70f4-8fa7-4575-9eb4-12d9f2a84bb4 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.127599 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec70f4-8fa7-4575-9eb4-12d9f2a84bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.127923 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec70f4-8fa7-4575-9eb4-12d9f2a84bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.148781 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c6ec70f4-8fa7-4575-9eb4-12d9f2a84bb4 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:00.150030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c6ec70f4-8fa7-4575-9eb4-12d9f2a84bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:00.206586 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1439/2884] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:40:00.214128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0484539-6817-45af-bbc6-5dec4eb58bd7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.214412 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0484539-6817-45af-bbc6-5dec4eb58bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.214688 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0484539-6817-45af-bbc6-5dec4eb58bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.233798 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a0484539-6817-45af-bbc6-5dec4eb58bd7 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:00.235014 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a0484539-6817-45af-bbc6-5dec4eb58bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:00.242692 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1446/2885] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:40:00.247142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec70f4-8fa7-4575-9eb4-12d9f2a84bb4 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:00.256460 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec70f4-8fa7-4575-9eb4-12d9f2a84bb4 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:00.261660 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851c361-925a-4767-bfc4-61f63deabe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:00.262243 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c361-925a-4767-bfc4-61f63deabe3e admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.262519 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c361-925a-4767-bfc4-61f63deabe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.262786 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c361-925a-4767-bfc4-61f63deabe3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.263410 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851c361-925a-4767-bfc4-61f63deabe3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:00.263739 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851c361-925a-4767-bfc4-61f63deabe3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:00.265323 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851c361-925a-4767-bfc4-61f63deabe3e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:00.276630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1440/2886] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:00.280779 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0484539-6817-45af-bbc6-5dec4eb58bd7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:00.289675 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0484539-6817-45af-bbc6-5dec4eb58bd7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:00.294306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2891af8-279f-4d84-99ce-37fa95a11982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:00.294890 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2891af8-279f-4d84-99ce-37fa95a11982 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.295164 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2891af8-279f-4d84-99ce-37fa95a11982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.295430 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2891af8-279f-4d84-99ce-37fa95a11982 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.296168 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2891af8-279f-4d84-99ce-37fa95a11982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:00.298106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2891af8-279f-4d84-99ce-37fa95a11982 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:00.326839 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1447/2887] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:40:00 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:40:00.331008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851c361-925a-4767-bfc4-61f63deabe3e admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:00.340693 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851c361-925a-4767-bfc4-61f63deabe3e admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:00.395818 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ed9b7-8ef2-4fd9-beb7-ea7b43b7edbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:00.396464 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ed9b7-8ef2-4fd9-beb7-ea7b43b7edbc admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.396782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ed9b7-8ef2-4fd9-beb7-ea7b43b7edbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.397326 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ed9b7-8ef2-4fd9-beb7-ea7b43b7edbc admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.398004 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ed9b7-8ef2-4fd9-beb7-ea7b43b7edbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:00.399992 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ed9b7-8ef2-4fd9-beb7-ea7b43b7edbc admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:00.421381 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1441/2888] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:40:00 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:40:00.425631 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2891af8-279f-4d84-99ce-37fa95a11982 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:00.436282 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2891af8-279f-4d84-99ce-37fa95a11982 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:00.440771 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6d942-b024-45d6-9fdf-5118969f37d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:00.441362 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6d942-b024-45d6-9fdf-5118969f37d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.441636 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6d942-b024-45d6-9fdf-5118969f37d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.441903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6d942-b024-45d6-9fdf-5118969f37d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.442534 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6d942-b024-45d6-9fdf-5118969f37d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:00.444159 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6d942-b024-45d6-9fdf-5118969f37d3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:00.449204 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:00.455557 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1448/2889] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:40:00.459708 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ed9b7-8ef2-4fd9-beb7-ea7b43b7edbc admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:00.469680 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ed9b7-8ef2-4fd9-beb7-ea7b43b7edbc admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:00.474349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a28be-4fd8-4eef-add8-ba4413f3e34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:00.474921 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a28be-4fd8-4eef-add8-ba4413f3e34c admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.475202 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a28be-4fd8-4eef-add8-ba4413f3e34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.475474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a28be-4fd8-4eef-add8-ba4413f3e34c admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.476171 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a28be-4fd8-4eef-add8-ba4413f3e34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:00.477755 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a28be-4fd8-4eef-add8-ba4413f3e34c admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:00.488643 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1442/2890] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:40:00.492554 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6d942-b024-45d6-9fdf-5118969f37d3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:00.502808 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6d942-b024-45d6-9fdf-5118969f37d3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:00.507349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469f470-4783-410c-8cf8-83cda93f0413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:00.508015 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469f470-4783-410c-8cf8-83cda93f0413 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.508293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469f470-4783-410c-8cf8-83cda93f0413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.508564 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469f470-4783-410c-8cf8-83cda93f0413 admin admin] PATH_INFO: `/v3/projects/2fd7aa7a292f4935a7314d2756f5646f/users/4b7c41e00a82443e8f35e5f2dbfa7e43/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.522737 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469f470-4783-410c-8cf8-83cda93f0413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd7aa7a292f4935a7314d2756f5646f, user_id=4b7c41e00a82443e8f35e5f2dbfa7e43, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:00.525595 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469f470-4783-410c-8cf8-83cda93f0413 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:00.538790 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1449/2891] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:40:00 2022] PUT /identity/v3/projects/2fd7aa7a292f4935a7314d2756f5646f/users/4b7c41e00a82443e8f35e5f2dbfa7e43/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:40:00.550490 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7025817-3c0a-46be-9b3a-c8660c1c4841 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.550778 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7025817-3c0a-46be-9b3a-c8660c1c4841 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.551051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7025817-3c0a-46be-9b3a-c8660c1c4841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.578772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-c7025817-3c0a-46be-9b3a-c8660c1c4841 None None] MFA Rules not processed for user `4b7c41e00a82443e8f35e5f2dbfa7e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:00.580096 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-c7025817-3c0a-46be-9b3a-c8660c1c4841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:00.613639 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:00.645703 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1443/2892] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:40:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:40:00.657697 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469f470-4783-410c-8cf8-83cda93f0413 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:00.659414 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b469f470-4783-410c-8cf8-83cda93f0413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:00.737860 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469f470-4783-410c-8cf8-83cda93f0413 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:00.741485 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8667091-9048-453d-a800-1f3c3113d2ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd61407406ef242a78030f79532fe2247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:00.742090 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8667091-9048-453d-a800-1f3c3113d2ad service glance] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.742366 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8667091-9048-453d-a800-1f3c3113d2ad service glance] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.742637 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8667091-9048-453d-a800-1f3c3113d2ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.750777 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8667091-9048-453d-a800-1f3c3113d2ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:00.752226 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8667091-9048-453d-a800-1f3c3113d2ad service glance] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:00.760802 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1450/2893] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:40:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:40:10.642856 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=12 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:10.756860 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=45,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:25.368089 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7025817-3c0a-46be-9b3a-c8660c1c4841 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:25.429634 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7025817-3c0a-46be-9b3a-c8660c1c4841 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:25.434122 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834d5dc-c9db-43cb-9167-95d1eb1dae01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:25.434738 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834d5dc-c9db-43cb-9167-95d1eb1dae01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:25.435019 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834d5dc-c9db-43cb-9167-95d1eb1dae01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:25.435295 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834d5dc-c9db-43cb-9167-95d1eb1dae01 admin admin] PATH_INFO: `/v3/users/4b7c41e00a82443e8f35e5f2dbfa7e43` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:25.436428 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834d5dc-c9db-43cb-9167-95d1eb1dae01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b7c41e00a82443e8f35e5f2dbfa7e43)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:25.438335 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834d5dc-c9db-43cb-9167-95d1eb1dae01 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:25.482911 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7834d5dc-c9db-43cb-9167-95d1eb1dae01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b7c41e00a82443e8f35e5f2dbfa7e43'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:25.495189 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7834d5dc-c9db-43cb-9167-95d1eb1dae01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b7c41e00a82443e8f35e5f2dbfa7e43'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:25.502756 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7834d5dc-c9db-43cb-9167-95d1eb1dae01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b7c41e00a82443e8f35e5f2dbfa7e43'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:25.503409 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7834d5dc-c9db-43cb-9167-95d1eb1dae01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b7c41e00a82443e8f35e5f2dbfa7e43'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:25.510205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-7834d5dc-c9db-43cb-9167-95d1eb1dae01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b7c41e00a82443e8f35e5f2dbfa7e43'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:25.516070 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1444/2894] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:40:25 2022] DELETE /identity/v3/users/4b7c41e00a82443e8f35e5f2dbfa7e43 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:40:25.520618 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402994b-e5b8-4b06-8242-65808e0b3dc8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:25.521111 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402994b-e5b8-4b06-8242-65808e0b3dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:25.521222 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402994b-e5b8-4b06-8242-65808e0b3dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:25.540016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5402994b-e5b8-4b06-8242-65808e0b3dc8 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:25.541216 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5402994b-e5b8-4b06-8242-65808e0b3dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:25.597946 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1451/2895] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:40:25.607486 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834d5dc-c9db-43cb-9167-95d1eb1dae01 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:25.608982 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-7834d5dc-c9db-43cb-9167-95d1eb1dae01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:25.665273 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834d5dc-c9db-43cb-9167-95d1eb1dae01 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:25.669062 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e4b93-3227-4f33-addc-f8d84a4b1120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:25.669655 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4b93-3227-4f33-addc-f8d84a4b1120 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:25.669932 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4b93-3227-4f33-addc-f8d84a4b1120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:25.670204 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4b93-3227-4f33-addc-f8d84a4b1120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:25.679580 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e4b93-3227-4f33-addc-f8d84a4b1120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:25.681009 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e4b93-3227-4f33-addc-f8d84a4b1120 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:25.690255 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1445/2896] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:40:25.952267 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402994b-e5b8-4b06-8242-65808e0b3dc8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:25.953739 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5402994b-e5b8-4b06-8242-65808e0b3dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:25.966849 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402994b-e5b8-4b06-8242-65808e0b3dc8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:25.972140 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d1844-09e1-47b7-aa87-274949896472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:25.973467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d1844-09e1-47b7-aa87-274949896472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:25.973756 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d1844-09e1-47b7-aa87-274949896472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:25.974030 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d1844-09e1-47b7-aa87-274949896472 admin admin] PATH_INFO: `/v3/projects/2fd7aa7a292f4935a7314d2756f5646f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:25.975149 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d1844-09e1-47b7-aa87-274949896472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd7aa7a292f4935a7314d2756f5646f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:25.977078 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d1844-09e1-47b7-aa87-274949896472 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:26.024467 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c92d1844-09e1-47b7-aa87-274949896472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd7aa7a292f4935a7314d2756f5646f', 'request_id': 'req-c92d1844-09e1-47b7-aa87-274949896472'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:26.029926 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c92d1844-09e1-47b7-aa87-274949896472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd7aa7a292f4935a7314d2756f5646f', 'request_id': 'req-c92d1844-09e1-47b7-aa87-274949896472'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:26.037868 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c92d1844-09e1-47b7-aa87-274949896472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd7aa7a292f4935a7314d2756f5646f', 'request_id': 'req-c92d1844-09e1-47b7-aa87-274949896472'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:26.048691 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c92d1844-09e1-47b7-aa87-274949896472 admin admin] The token cache is being invalidate because project 2fd7aa7a292f4935a7314d2756f5646f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:40:26.048977 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c92d1844-09e1-47b7-aa87-274949896472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:26.049617 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c92d1844-09e1-47b7-aa87-274949896472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:26.051082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1452/2897] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:40:25 2022] DELETE /identity/v3/projects/2fd7aa7a292f4935a7314d2756f5646f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:40:26.055834 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358f2fd-08b3-4307-8d6b-abfd9cb89c6c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:26.056120 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358f2fd-08b3-4307-8d6b-abfd9cb89c6c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:26.056420 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358f2fd-08b3-4307-8d6b-abfd9cb89c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:26.075245 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-3358f2fd-08b3-4307-8d6b-abfd9cb89c6c None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:26.076476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-3358f2fd-08b3-4307-8d6b-abfd9cb89c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:26.093017 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:26.137063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1446/2898] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:40:26.145153 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d407e-eff3-4c87-8009-b0dc555c8557 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:26.145437 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d407e-eff3-4c87-8009-b0dc555c8557 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:26.145711 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d407e-eff3-4c87-8009-b0dc555c8557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:26.165698 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-882d407e-eff3-4c87-8009-b0dc555c8557 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:26.166882 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-882d407e-eff3-4c87-8009-b0dc555c8557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:26.174567 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1453/2899] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:40:26.178664 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358f2fd-08b3-4307-8d6b-abfd9cb89c6c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:26.188133 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358f2fd-08b3-4307-8d6b-abfd9cb89c6c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:26.192669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c28bc-1e8b-4dcc-8664-82a9313f279e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:26.193251 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c28bc-1e8b-4dcc-8664-82a9313f279e admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:26.193528 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c28bc-1e8b-4dcc-8664-82a9313f279e admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:26.193798 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c28bc-1e8b-4dcc-8664-82a9313f279e admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:26.194429 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c28bc-1e8b-4dcc-8664-82a9313f279e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:26.194729 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c28bc-1e8b-4dcc-8664-82a9313f279e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:26.196356 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c28bc-1e8b-4dcc-8664-82a9313f279e admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:26.207972 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1447/2900] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:26.212004 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d407e-eff3-4c87-8009-b0dc555c8557 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:26.221507 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d407e-eff3-4c87-8009-b0dc555c8557 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:26.226128 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc45b72-f545-4c8f-a95b-9a73604f8510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:26.226704 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc45b72-f545-4c8f-a95b-9a73604f8510 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:26.226981 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc45b72-f545-4c8f-a95b-9a73604f8510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:26.227252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc45b72-f545-4c8f-a95b-9a73604f8510 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:26.227982 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc45b72-f545-4c8f-a95b-9a73604f8510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:26.229847 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc45b72-f545-4c8f-a95b-9a73604f8510 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:26.255930 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1454/2901] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:40:26 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:40:26.259900 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c28bc-1e8b-4dcc-8664-82a9313f279e admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:26.270068 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c28bc-1e8b-4dcc-8664-82a9313f279e admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:26.323474 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97a6a1-7056-4c2c-9b91-c64db9aec0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:26.324090 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97a6a1-7056-4c2c-9b91-c64db9aec0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:26.324366 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97a6a1-7056-4c2c-9b91-c64db9aec0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:26.324640 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97a6a1-7056-4c2c-9b91-c64db9aec0d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:26.325266 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c97a6a1-7056-4c2c-9b91-c64db9aec0d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:26.327249 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c97a6a1-7056-4c2c-9b91-c64db9aec0d9 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:26.349354 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1448/2902] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:40:26 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:40:26.353393 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc45b72-f545-4c8f-a95b-9a73604f8510 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:26.363501 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc45b72-f545-4c8f-a95b-9a73604f8510 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:26.368053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b7369-76da-4dd2-8389-80f938eda545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:26.368639 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b7369-76da-4dd2-8389-80f938eda545 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:26.368913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b7369-76da-4dd2-8389-80f938eda545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:26.369185 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b7369-76da-4dd2-8389-80f938eda545 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:26.369816 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b7369-76da-4dd2-8389-80f938eda545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:26.371381 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b7369-76da-4dd2-8389-80f938eda545 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:26.381510 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1455/2903] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:40:26.385241 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97a6a1-7056-4c2c-9b91-c64db9aec0d9 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:26.395784 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97a6a1-7056-4c2c-9b91-c64db9aec0d9 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:26.400630 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36681cb6-f0dd-4ceb-8ce1-512160bec107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:26.401209 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36681cb6-f0dd-4ceb-8ce1-512160bec107 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:26.401489 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36681cb6-f0dd-4ceb-8ce1-512160bec107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:26.401763 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36681cb6-f0dd-4ceb-8ce1-512160bec107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:26.402404 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36681cb6-f0dd-4ceb-8ce1-512160bec107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:26.404058 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36681cb6-f0dd-4ceb-8ce1-512160bec107 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:26.414351 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1449/2904] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:40:26.418293 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b7369-76da-4dd2-8389-80f938eda545 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:26.428184 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b7369-76da-4dd2-8389-80f938eda545 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:26.432811 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a120b9e-d6ec-4531-9f8f-3dbc374b3338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:26.433447 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a120b9e-d6ec-4531-9f8f-3dbc374b3338 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:26.433728 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a120b9e-d6ec-4531-9f8f-3dbc374b3338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:26.434005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a120b9e-d6ec-4531-9f8f-3dbc374b3338 admin admin] PATH_INFO: `/v3/projects/4876c3b1a7ed46ea92c95a477f17cbda/users/bb8c08ad499d47ddb114938c9c322369/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:26.448943 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a120b9e-d6ec-4531-9f8f-3dbc374b3338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4876c3b1a7ed46ea92c95a477f17cbda, user_id=bb8c08ad499d47ddb114938c9c322369, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:26.451614 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a120b9e-d6ec-4531-9f8f-3dbc374b3338 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:26.465003 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1456/2905] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:40:26 2022] PUT /identity/v3/projects/4876c3b1a7ed46ea92c95a477f17cbda/users/bb8c08ad499d47ddb114938c9c322369/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:40:26.476314 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee01a-022d-4ef3-9c93-640f1b93a8d2 None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:26.476603 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee01a-022d-4ef3-9c93-640f1b93a8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:26.476879 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee01a-022d-4ef3-9c93-640f1b93a8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:26.504975 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-387ee01a-022d-4ef3-9c93-640f1b93a8d2 None None] MFA Rules not processed for user `bb8c08ad499d47ddb114938c9c322369`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:26.506188 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-387ee01a-022d-4ef3-9c93-640f1b93a8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:26.568463 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1450/2906] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:40:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:40:26.579640 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a120b9e-d6ec-4531-9f8f-3dbc374b3338 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:26.581946 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3a120b9e-d6ec-4531-9f8f-3dbc374b3338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:26.604124 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:26.641825 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a120b9e-d6ec-4531-9f8f-3dbc374b3338 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:26.646137 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cddf15-86e3-4ff7-8e0b-998e038bb18e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:26.646683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cddf15-86e3-4ff7-8e0b-998e038bb18e service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:26.646965 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cddf15-86e3-4ff7-8e0b-998e038bb18e service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:26.647243 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cddf15-86e3-4ff7-8e0b-998e038bb18e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:26.655706 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cddf15-86e3-4ff7-8e0b-998e038bb18e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:26.657131 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cddf15-86e3-4ff7-8e0b-998e038bb18e service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:26.665915 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1457/2907] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:40:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:40:36.565674 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:36.661824 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=17 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:59.593132 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ee01a-022d-4ef3-9c93-640f1b93a8d2 None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:59.654556 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ee01a-022d-4ef3-9c93-640f1b93a8d2 None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:59.659076 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c7586-9269-463d-8ab2-6734c21a2094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:59.659726 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c7586-9269-463d-8ab2-6734c21a2094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:59.660024 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c7586-9269-463d-8ab2-6734c21a2094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:59.660280 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c7586-9269-463d-8ab2-6734c21a2094 admin admin] PATH_INFO: `/v3/users/bb8c08ad499d47ddb114938c9c322369` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:59.661380 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c7586-9269-463d-8ab2-6734c21a2094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb8c08ad499d47ddb114938c9c322369)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:59.663341 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c7586-9269-463d-8ab2-6734c21a2094 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:59.708417 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-701c7586-9269-463d-8ab2-6734c21a2094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb8c08ad499d47ddb114938c9c322369'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:59.719794 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-701c7586-9269-463d-8ab2-6734c21a2094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb8c08ad499d47ddb114938c9c322369'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:59.726996 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-701c7586-9269-463d-8ab2-6734c21a2094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb8c08ad499d47ddb114938c9c322369'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:59.727687 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-701c7586-9269-463d-8ab2-6734c21a2094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb8c08ad499d47ddb114938c9c322369'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:59.735938 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-701c7586-9269-463d-8ab2-6734c21a2094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb8c08ad499d47ddb114938c9c322369'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:40:59.742221 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1451/2908] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:40:59 2022] DELETE /identity/v3/users/bb8c08ad499d47ddb114938c9c322369 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:40:59.747275 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d15a-1c62-4d35-a434-c991db5f6bc0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:59.747862 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d15a-1c62-4d35-a434-c991db5f6bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:59.747992 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d15a-1c62-4d35-a434-c991db5f6bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:59.768029 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4032d15a-1c62-4d35-a434-c991db5f6bc0 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:59.769302 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4032d15a-1c62-4d35-a434-c991db5f6bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:59.827296 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1458/2909] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:40:59.836714 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c7586-9269-463d-8ab2-6734c21a2094 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:59.838096 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-701c7586-9269-463d-8ab2-6734c21a2094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:59.894110 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c7586-9269-463d-8ab2-6734c21a2094 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:59.897782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367a528-220b-4262-a180-55b6b479821d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:59.898650 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a528-220b-4262-a180-55b6b479821d service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:59.898928 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a528-220b-4262-a180-55b6b479821d service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:59.899205 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a528-220b-4262-a180-55b6b479821d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:59.908105 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367a528-220b-4262-a180-55b6b479821d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:59.909548 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367a528-220b-4262-a180-55b6b479821d service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:59.919100 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1452/2910] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:41:00.185203 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032d15a-1c62-4d35-a434-c991db5f6bc0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.186666 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4032d15a-1c62-4d35-a434-c991db5f6bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:00.198669 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032d15a-1c62-4d35-a434-c991db5f6bc0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.203836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c937c5d-1632-4f49-8276-aaaca3024844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.204465 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c937c5d-1632-4f49-8276-aaaca3024844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.204748 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c937c5d-1632-4f49-8276-aaaca3024844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.205023 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c937c5d-1632-4f49-8276-aaaca3024844 admin admin] PATH_INFO: `/v3/projects/4876c3b1a7ed46ea92c95a477f17cbda` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.206149 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c937c5d-1632-4f49-8276-aaaca3024844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4876c3b1a7ed46ea92c95a477f17cbda)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.212903 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c937c5d-1632-4f49-8276-aaaca3024844 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.260349 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3c937c5d-1632-4f49-8276-aaaca3024844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4876c3b1a7ed46ea92c95a477f17cbda', 'request_id': 'req-3c937c5d-1632-4f49-8276-aaaca3024844'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:00.265733 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3c937c5d-1632-4f49-8276-aaaca3024844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4876c3b1a7ed46ea92c95a477f17cbda', 'request_id': 'req-3c937c5d-1632-4f49-8276-aaaca3024844'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:00.273053 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3c937c5d-1632-4f49-8276-aaaca3024844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4876c3b1a7ed46ea92c95a477f17cbda', 'request_id': 'req-3c937c5d-1632-4f49-8276-aaaca3024844'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:00.284161 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3c937c5d-1632-4f49-8276-aaaca3024844 admin admin] The token cache is being invalidate because project 4876c3b1a7ed46ea92c95a477f17cbda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:41:00.284474 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3c937c5d-1632-4f49-8276-aaaca3024844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:00.285106 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3c937c5d-1632-4f49-8276-aaaca3024844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:00.287083 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1459/2911] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:41:00 2022] DELETE /identity/v3/projects/4876c3b1a7ed46ea92c95a477f17cbda => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:41:00.292063 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77610ce9-28a0-4b1a-a4bf-2f80253af81e None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.292349 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77610ce9-28a0-4b1a-a4bf-2f80253af81e None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.292623 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77610ce9-28a0-4b1a-a4bf-2f80253af81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.313200 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-77610ce9-28a0-4b1a-a4bf-2f80253af81e None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:00.314382 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-77610ce9-28a0-4b1a-a4bf-2f80253af81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:00.330385 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:00.374165 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1453/2912] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:41:00.381895 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e8658-002b-4201-bae4-c8d491a96d46 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.382178 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e8658-002b-4201-bae4-c8d491a96d46 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.382455 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e8658-002b-4201-bae4-c8d491a96d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.401963 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-eb1e8658-002b-4201-bae4-c8d491a96d46 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:00.403166 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eb1e8658-002b-4201-bae4-c8d491a96d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:00.410897 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1460/2913] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:41:00.415490 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77610ce9-28a0-4b1a-a4bf-2f80253af81e None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.425095 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77610ce9-28a0-4b1a-a4bf-2f80253af81e None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.430254 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10651c5f-d4cd-405f-af21-9830f1a2dad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.430838 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10651c5f-d4cd-405f-af21-9830f1a2dad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.431116 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10651c5f-d4cd-405f-af21-9830f1a2dad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.431385 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10651c5f-d4cd-405f-af21-9830f1a2dad8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.432052 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10651c5f-d4cd-405f-af21-9830f1a2dad8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:00.432355 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10651c5f-d4cd-405f-af21-9830f1a2dad8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.433927 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10651c5f-d4cd-405f-af21-9830f1a2dad8 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.445008 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1454/2914] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Sep 12 23:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:00.449099 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e8658-002b-4201-bae4-c8d491a96d46 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.458044 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e8658-002b-4201-bae4-c8d491a96d46 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.463298 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a641ca-34cb-411b-a32f-7207be8643e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.463913 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a641ca-34cb-411b-a32f-7207be8643e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.464191 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a641ca-34cb-411b-a32f-7207be8643e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.464461 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a641ca-34cb-411b-a32f-7207be8643e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.465082 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a641ca-34cb-411b-a32f-7207be8643e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.466954 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a641ca-34cb-411b-a32f-7207be8643e2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.494139 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1461/2915] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Sep 12 23:41:00 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:41:00.498158 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10651c5f-d4cd-405f-af21-9830f1a2dad8 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.507768 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10651c5f-d4cd-405f-af21-9830f1a2dad8 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.562272 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d407df2-25db-45e2-a661-3a46dfd67e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.562875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d407df2-25db-45e2-a661-3a46dfd67e2d admin admin] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.563156 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d407df2-25db-45e2-a661-3a46dfd67e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.563470 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d407df2-25db-45e2-a661-3a46dfd67e2d admin admin] PATH_INFO: `/v3/users` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.564120 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d407df2-25db-45e2-a661-3a46dfd67e2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.566050 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d407df2-25db-45e2-a661-3a46dfd67e2d admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.587051 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1455/2916] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Sep 12 23:41:00 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:41:00.591183 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a641ca-34cb-411b-a32f-7207be8643e2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.601331 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a641ca-34cb-411b-a32f-7207be8643e2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.605878 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473870e5-3219-48f5-a9fc-fc41ee721516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.606463 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473870e5-3219-48f5-a9fc-fc41ee721516 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.606738 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473870e5-3219-48f5-a9fc-fc41ee721516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.607005 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473870e5-3219-48f5-a9fc-fc41ee721516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.607677 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473870e5-3219-48f5-a9fc-fc41ee721516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.609256 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473870e5-3219-48f5-a9fc-fc41ee721516 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.619617 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1462/2917] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:41:00.623393 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d407df2-25db-45e2-a661-3a46dfd67e2d admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.634015 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d407df2-25db-45e2-a661-3a46dfd67e2d admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.638609 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05842e6-d32c-4d20-a539-d66fb7ffbd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.639196 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05842e6-d32c-4d20-a539-d66fb7ffbd2a admin admin] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.639476 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05842e6-d32c-4d20-a539-d66fb7ffbd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.639782 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05842e6-d32c-4d20-a539-d66fb7ffbd2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.640416 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05842e6-d32c-4d20-a539-d66fb7ffbd2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.641998 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05842e6-d32c-4d20-a539-d66fb7ffbd2a admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.652669 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1456/2918] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Sep 12 23:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:41:00.656705 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473870e5-3219-48f5-a9fc-fc41ee721516 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.666389 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473870e5-3219-48f5-a9fc-fc41ee721516 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.670905 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbe8b9-438d-4165-9997-ec63f3142848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.671536 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbe8b9-438d-4165-9997-ec63f3142848 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.671846 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbe8b9-438d-4165-9997-ec63f3142848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.672116 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbe8b9-438d-4165-9997-ec63f3142848 admin admin] PATH_INFO: `/v3/projects/bc282e60ce4f4420b3d22cbd39e3a828/users/96f7cd34fe524521a4e2b96fd5b401ad/roles/ebbe7434d1ec4bc4ba71737f55987dd9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.686661 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbe8b9-438d-4165-9997-ec63f3142848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc282e60ce4f4420b3d22cbd39e3a828, user_id=96f7cd34fe524521a4e2b96fd5b401ad, role_id=ebbe7434d1ec4bc4ba71737f55987dd9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.689309 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbe8b9-438d-4165-9997-ec63f3142848 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.702246 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1463/2919] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Sep 12 23:41:00 2022] PUT /identity/v3/projects/bc282e60ce4f4420b3d22cbd39e3a828/users/96f7cd34fe524521a4e2b96fd5b401ad/roles/ebbe7434d1ec4bc4ba71737f55987dd9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:41:00.713508 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61174d5-ccfd-4eed-958f-823fd08fb38c None None] REQUEST_METHOD: `POST` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.713799 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61174d5-ccfd-4eed-958f-823fd08fb38c None None] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.714078 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61174d5-ccfd-4eed-958f-823fd08fb38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.741715 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.auth.core [None req-b61174d5-ccfd-4eed-958f-823fd08fb38c None None] MFA Rules not processed for user `96f7cd34fe524521a4e2b96fd5b401ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:00.742924 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-b61174d5-ccfd-4eed-958f-823fd08fb38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:00.805896 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1457/2920] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:41:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:41:00.816995 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbe8b9-438d-4165-9997-ec63f3142848 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.818734 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1fdbe8b9-438d-4165-9997-ec63f3142848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:00.839876 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:00.877016 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbe8b9-438d-4165-9997-ec63f3142848 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.880659 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdb1df-6b3a-4fd7-a554-c68d35c17102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5b8828077594349a93c7a842c0f15da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.881252 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdb1df-6b3a-4fd7-a554-c68d35c17102 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.881529 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdb1df-6b3a-4fd7-a554-c68d35c17102 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.881804 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdb1df-6b3a-4fd7-a554-c68d35c17102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.889867 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdb1df-6b3a-4fd7-a554-c68d35c17102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.891274 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdb1df-6b3a-4fd7-a554-c68d35c17102 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.900132 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1464/2921] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:41:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:41:00.925111 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61174d5-ccfd-4eed-958f-823fd08fb38c None None] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.943979 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:00.988405 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61174d5-ccfd-4eed-958f-823fd08fb38c None None] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.993677 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272973f-d10f-4b42-831f-f126f9e642a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.994301 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272973f-d10f-4b42-831f-f126f9e642a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.994584 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272973f-d10f-4b42-831f-f126f9e642a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.994861 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272973f-d10f-4b42-831f-f126f9e642a5 admin admin] PATH_INFO: `/v3/users/96f7cd34fe524521a4e2b96fd5b401ad` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.996037 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272973f-d10f-4b42-831f-f126f9e642a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96f7cd34fe524521a4e2b96fd5b401ad)` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.997948 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272973f-d10f-4b42-831f-f126f9e642a5 admin admin] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:01.042644 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9272973f-d10f-4b42-831f-f126f9e642a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96f7cd34fe524521a4e2b96fd5b401ad'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:01.055471 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9272973f-d10f-4b42-831f-f126f9e642a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96f7cd34fe524521a4e2b96fd5b401ad'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:01.063256 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9272973f-d10f-4b42-831f-f126f9e642a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96f7cd34fe524521a4e2b96fd5b401ad'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:01.064030 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9272973f-d10f-4b42-831f-f126f9e642a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96f7cd34fe524521a4e2b96fd5b401ad'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:01.071003 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.notifications [None req-9272973f-d10f-4b42-831f-f126f9e642a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96f7cd34fe524521a4e2b96fd5b401ad'} {{(pid=81416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:01.076875 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1458/2922] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Sep 12 23:41:00 2022] DELETE /identity/v3/users/96f7cd34fe524521a4e2b96fd5b401ad => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:41:01.081251 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd6ef7-4327-4397-971d-c6057379fca6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:01.081561 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd6ef7-4327-4397-971d-c6057379fca6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:01.081836 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd6ef7-4327-4397-971d-c6057379fca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:01.100746 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-cecd6ef7-4327-4397-971d-c6057379fca6 None None] MFA Rules not processed for user `318df21351594db286347c68d09ec473`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:01.101861 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cecd6ef7-4327-4397-971d-c6057379fca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:01.158306 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1465/2923] 10.222.0.27 () {58 vars in 958 bytes} [Mon Sep 12 23:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:41:01.168398 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272973f-d10f-4b42-831f-f126f9e642a5 admin admin] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:01.169772 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-9272973f-d10f-4b42-831f-f126f9e642a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:01.225235 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272973f-d10f-4b42-831f-f126f9e642a5 admin admin] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:01.228933 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fccb6-cbdf-4d22-a975-ceeebe494f41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc6e63e9fc3447d5a894a777a98fe773', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:01.229591 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fccb6-cbdf-4d22-a975-ceeebe494f41 service neutron] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:01.229872 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fccb6-cbdf-4d22-a975-ceeebe494f41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:01.230142 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fccb6-cbdf-4d22-a975-ceeebe494f41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:01.238937 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983fccb6-cbdf-4d22-a975-ceeebe494f41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:01.240401 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983fccb6-cbdf-4d22-a975-ceeebe494f41 service neutron] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:01.249525 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1459/2924] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:41:01.508157 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd6ef7-4327-4397-971d-c6057379fca6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:01.509577 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cecd6ef7-4327-4397-971d-c6057379fca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:01.521946 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd6ef7-4327-4397-971d-c6057379fca6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:01.526777 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '318df21351594db286347c68d09ec473', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b515d316021473bab5135bffb78aaff', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:01.527401 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:01.527699 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:01.527973 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f admin admin] PATH_INFO: `/v3/projects/bc282e60ce4f4420b3d22cbd39e3a828` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:01.529081 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc282e60ce4f4420b3d22cbd39e3a828)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:01.530966 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:01.576324 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc282e60ce4f4420b3d22cbd39e3a828', 'request_id': 'req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:01.581683 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc282e60ce4f4420b3d22cbd39e3a828', 'request_id': 'req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:01.589465 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc282e60ce4f4420b3d22cbd39e3a828', 'request_id': 'req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:01.600641 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f admin admin] The token cache is being invalidate because project bc282e60ce4f4420b3d22cbd39e3a828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:41:01.600933 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:01.601569 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c6073d28-c0f0-45fd-944e-90ec6ab5e48f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:41:01.603097 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1466/2925] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Sep 12 23:41:01 2022] DELETE /identity/v3/projects/bc282e60ce4f4420b3d22cbd39e3a828 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:41:11.245542 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:11.597667 dv-cinder-852009-7-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:18.261485 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fccb6-cbdf-4d22-a975-ceeebe494f41 service neutron] Authenticating user token {{(pid=81416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:18.263638 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-983fccb6-cbdf-4d22-a975-ceeebe494f41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:18.330829 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fccb6-cbdf-4d22-a975-ceeebe494f41 service neutron] Validating token access rules against request {{(pid=81416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:18.387904 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e6246-a0ca-4ae9-b5f8-02176cc80db5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b3b0a0317b14e29a32eb6ba6b89c2aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32bbb936501846e8914acf07890c2891', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:18.388506 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e6246-a0ca-4ae9-b5f8-02176cc80db5 service placement] REQUEST_METHOD: `GET` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:18.388789 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e6246-a0ca-4ae9-b5f8-02176cc80db5 service placement] SCRIPT_NAME: `/identity` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:18.389064 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e6246-a0ca-4ae9-b5f8-02176cc80db5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:18.390757 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.fernet_utils [None req-ec1e6246-a0ca-4ae9-b5f8-02176cc80db5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:18.402533 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e6246-a0ca-4ae9-b5f8-02176cc80db5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:18.404161 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e6246-a0ca-4ae9-b5f8-02176cc80db5 service placement] RBAC: Authorization granted {{(pid=81416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:18.414615 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: [pid: 81416|app: 0|req: 1460/2926] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Sep 12 23:41:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:41:28.409616 dv-cinder-852009-7-iscsi devstack@keystone.service[81416]: DEBUG dbcounter [-] [81416] Writing DB stats keystone:SELECT=58 {{(pid=81416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}